Comments on Draft Final Proposal

1. Please provide a summary of your organization’s position on the draft final proposal:

The Bonneville Power Administration (Bonneville)[1] appreciates the opportunity to submit comments on CAISO’s EIM Resource Sufficiency Evaluation (RSE) Enhancements Draft Final Proposal dated October 6, 2021. Bonneville reiterates the importance of the EIM RSE as a foundational market design element of the EIM. EIM Entities, including Bonneville, agreed to enter the EIM with the understanding that the EIM RSE was intended to ensure an EIM Entity is able to meet their demand with their own net-supply prior to engaging in transfers with other BAAs through the EIM in the real-time market.[2]  Without an accurate test, the EIM RSE cannot perform the function it was intended to achieve. And while Bonneville believes preventing leaning requires more robust failure consequences, the accuracy and transparency of the test is the first critical step and we are concerned it will not be achieved within the construct of this Draft Final Proposal as currently written.

Two measures of relative change in EIM RSE accuracy are of concern to Bonneville.  First, and most concerning, comparing performance of the EIM RSE in the summer of 2021 with the summer of 2020 under similarly challenging operating conditions shows the CAISO BAA both passing and failing the EIM RSE during a stage 2 energy emergency alert:

Without further enhancement to the Draft Final Proposal now, we are concerned performance of the EIM RSE in the summer of 2022 will yield similarly inaccurate EIM RSE results that appear to contradict energy emergency declarations.

Second, comparing the frequency of Bid Range Capacity Test (BRCT) failures before and after the implementation of the additional uncertainty requirement on June 16, 2021 reveals a meaningful increase in the number of test failures.  Bonneville appreciates CAISO’s attempt to better account for uncertainty, as it contributed to some of the issues in 2020, and Bonneville recognizes that an increase in failures is an expected outcome of changes in the uncertainty requirement. However, further evaluation and explanation of the current implementation to further refine the requirement should be pursued in Phase 1.

While the changes the CAISO has proposed in the Draft Final Proposal provide minor improvements, these indicators reinforce our belief that the proposed changes do not go far enough to achieve the objectives of Phase 1 of this initiative – accuracy and transparency of the EIM RSE – and appears to perpetuate problems with CAISO passing the test during periods of inadequacy while imposing more stringent requirements on EIM Entities.  We observe the lack of changes made between the Straw Proposal and the Draft Final Proposal and cannot support the Draft Final Proposal without further policy enhancements specifically aimed at improving the accuracy of the EIM RSE’s ability to assess the operating capability and obligations of each Balancing Authority Area (BAA).

The four key areas that should be addressed to improve the accuracy of the EIM RSE are:

• Capacity should be capable of performing in real-time. The capacity test should only count resources that are truly available for the operating interval(s) being evaluated. We struggle to understand the rationale for including resources made available in the Short-Term Unit Commitment (STUC) when those resources are not available for the operating hour being evaluated. Bonneville continues to recommend that the capacity test only count resources that have a start-up and minimum run time no longer than what can be started by the Real-Time Unit Commitment (RTUC) process (approximately 2 hours out).
• Include systemic load conformance. The EIM RSE should incorporate load conformance that is systemic, frequent and of high magnitude and this enhancement should be made as part of Phase 1 of this initiative.  Based on DMM analysis to date, only the CAISO BAA fits this description, and thus Bonneville believes the most compelling approach is to add the CAISO BAA’s upward imbalance conformance to its EIM RSE requirement.
• Disable intertie uncertainty. As noted by the CAISO, the Intertie Deviation Uncertainty (IDU) calculation using a 90 day look back is not an accurate indicator of future expected intertie uncertainty. Recent analysis indicates use of the inaccurate intertie uncertainty is contributing to 50% of the EIM Entities’ EIM RSE test failures in the April to August period in 2021.  Bonneville believes the IDU element should be disabled in Phase 1 in order to rectify calculation errors and determine in Phase 2 whether a re-designed IDU element should significantly reduce the confidence intervals the CAISO suggested, increase the sample size, eliminate outliers or permanently disable the IDU element of the BRCT.
• Expedite use of quantile regression for VER and load uncertainty and commit to review additional enhancements. The calculation of uncertainty plays a significant role in the accuracy of the EIM RSE. CAISO has delayed implementation of a quantile regression approach multiple times. Bonneville requests that CAISO provide a clear commitment to a specific implementation date, to a thorough evaluation of the effectiveness of the quantile regression approach, and to further enhancements if needed.

Bonneville urges the CAISO to prioritize these enhancements to the EIM RSE in Phase 1 of this initiative.

[1] Bonneville is a federal power marketing administration within the U.S. Department of Energy that markets electric power from 31 federal hydroelectric projects and some non-federal projects in the Pacific Northwest with a nameplate capacity of 22,500 MW. Bonneville currently supplies around 30 percent of the power consumed in the Northwest. Bonneville also operates 15,000 miles of high voltage transmission that interconnects most of the other transmission systems in the Northwest with Canada and California. Bonneville is obligated by statute to serve Northwest municipalities, public utility districts, cooperatives and then other regional entities prior to selling power out of the region.

[2] P. 5 of CAISO’s EIM Resource Sufficiency Evaluation Enhancements Draft Final Proposal

2. Please provide your organization's comments on the proposed modifications to the capacity test - intertemporal constraints proposal:

Bonneville continues to oppose the CAISO’s proposal to utilize the STUC – which commits and decommits resources 4.5 hours ahead of the operating hour – to determine whether a bid from a resource at that time defines participation in the real-time market even though the CAISO has long maintained a RTUC – which commits and decommits resources 2 hours ahead of the operating hour.  Bonneville appreciates the CAISO clearly laying out the 5 scenarios in earlier versions of the proposal and encourages the CAISO to consider suggestions made by Vistra and Bonneville during the October 6^th workshop and evaluate their effectiveness.  The suggestion to count offline capacity at a PMIN or minimum load level rather than a PMAX could be an appropriate step in the right direction and we encourage CAISO to update / add to the 5 scenarios in the Draft Final Proposal to further describe any proposed changes related to the credited capacity.

More broadly, Bonneville encourages the CAISO to have a more robust discussion of the conflicting principles of market efficiency and prevention of leaning inherent in this intertemporal constraints policy proposal.  The CAISO appears to be favoring market efficiency while also simultaneously violating the prevention of leaning principle based on analysis provided by the DMM in their September monthly report in which EIM transfers into the CAISO on July 9^th exceeded all measures of unloaded capacity (intertemporally constrained or not) available in the CAISO BAA.  While Bonneville acknowledges this was a challenging operational day owing to significant transmission derates due to the Bootleg fire, Bonneville encourages DMM and the CAISO to provide further analysis on the role of intertemporal constraints limiting the unloaded capacity available in the CAISO BAA to levels below the net EIM transfers into the CAISO BAA on other operating days.  Similar analyses should also be performed on EIM Entity BAAs and the EIM Area as a whole.

3. Please provide your organization's comments on the value of adding additional rules to the capacity test to more accurately count the capacity that while offline during the RSE, was offered into the real-time market for use:

Bonneville believes that the RTUC, at 2 hours ahead of the operating hour, more appropriately balances the principles of market efficiency and prevention of leaning, while also acknowledging there would be dispatch costs associated with adopting this proposal instead of the CAISO proposal for utilizing the STUC 4.5 hours out.  However, in light of the tight market conditions in the West, a 2021 summer season that did not experience a widespread heat event, growing penetrations of variable energy resources and the need to sustain resiliency in the face of increasing wildfires impacting transmission systems that facilitate market transfer between regions, Bonneville believes these stand-ready dispatch costs are both warranted and equitable.

4. Please provide your organization's comments on the consideration of storage resources by the RSE:

Bonneville appreciates the CAISO requesting comments regarding its proposal “in phase 1 to limit the counting of these resources to the capacity corresponding to their amount of charge at the time of the EIM RSE, plus any additional amount made available through energy bids to charge.”  This proposal raises several questions:

• What are the specific types of resources that are considered to be “storage resources” – only batteries and pumped hydro?
• What specific time and quantities is the CAISO considering?  Is the quantity a specific dispatch operating target or an actual state of charge?  Would this quantity change for each iteration of the test (ie. a different quantity for each of the three sequential EIM RSE test runs) or be a single quantity that would be the same in each of the three iterations of the test?
• We understand that the CAISO considering adding the incremental energy bids above the quantity in the previous bullet to the quantity in the previous bullet.  Is this understanding correct?  An example articulating the quantity to be counted would be helpful.

Bonneville would appreciate CAISO’s perspective on these questions.

5. Please provide your organization's comments on the proposed modifications to the flexible ramping test proposal:

Bonneville continues to support in concept the CAISO’s proposal to account for the quantity of any power balance constraint relaxation that is present in the market solution as part of the flexible ramping sufficiency test, for both the upward and downward requirements. This is an important design gap that we believe could be closed by such a policy enhancement.  While we appreciate the graphical representation included in the Draft Final Proposal, we request that the CAISO demonstrate that this change is sufficient to remove the potential for infeasible operating schedules to be used in the flexible ramping sufficiency test by evaluating how this change would have performed during one of the challenging operating days such as August 14/15, 2020 or July 9, 2021. In performing this evaluation, we also request that the CAISO provide the quantities of operator adjustments applicable in the RTPD interval used as the reference point for the hours being evaluated.

6. Please provide your organization's comments on the proposed modifications to the balancing test proposal:

Bonneville has no comments.

7. Please provide your organization's comments on the clarifications to the demand response proposal:

Bonneville specifically calls the CAISO’s attention to the EIM Entities’ comment on this topic and offers no further comment.

8. Please provide your organization's comments on the reporting and any additional metrics your organization believes may be valuable:

Bonneville continues to support enhanced reporting on the information proposed by the CAISO for each EIM participant with the following modifications:

• Unloaded capacity versus EIM transfers for all intervals during tight system conditions.  Reporting done by DMM on the CAISO BAA for July 9^th is a good starting point, but additional days and additional BAAs are required to support accuracy and transparency of the EIM RSE.
• Renewable forecast used in the EIM RSE versus renewable forecast used in real-time pre-dispatch versus renewable forecast used in real-time dispatch.  In particular, this will support data-driven policy development on the uncertainty element of the BRCT requirement and is directly responsive to commitments made in the CAISO tariff filing.
• Load forecast used in the EIM RSE versus renewable forecast used in real-time pre-dispatch versus renewable forecast used in real-time dispatch.  In particular, this will support data-driven policy development on the uncertainty element of the BRCT requirement and is directly responsive to commitments made in the CAISO tariff filing.

• An overall evaluation of whether the total RS requirements are accurate. In other words, does the overall level of capacity and flexibility required by the EIM RSE reflect actual observations within the targeted P95 confidence level?

Bonneville reiterates its recommendation that the DMM provide quarterly reporting to the EIM

Governing Body that includes a routine review of EIM RSE performance (i.e. pass/fails, quantities, etc.), EIM RSE accuracy metrics, and any issues and/or recommendations raised by DMM, the Governing Body Expert, and/or other stakeholders. Bonneville supports development of performance and accuracy metrics as proposed by the EIM Entities.

9. Please provide your organization’s comments on the level of EIM entity data granularity supporting the Department of Market Monitoring’s analysis that should be made publically available:

Data transparency is an important element in policy development.  As such, provision at the BAA level is appropriate to help pinpoint refinements to the existing methodology and / or formulate novel policy proposals.

10. Please provide your organization's comments on the potential to relax the under-scheduling tests requirement or consequences when demand response is used. If your organization see’s value in this approach, please provide additional detail on how you envision such relaxation being structured:

Bonneville supports the EIM Entities’ comment and has no further comment.

11. Please provide your organization's comments on the clarifications to the emergency actions proposal:

Bonneville supports in concept the CAISO’s proposal to limit the real-time market’s dispatch of additional energy transfers into a BAA when a BAA is under an energy emergency.  The Draft Final Proposal indicates that “a balancing authority area should be deemed resource insufficient in the event it is in an energy emergency and has resorted to arming firm load to meet reserve requirements.” [Emphasis added]  It goes on to say, “For example, the CAISO was in such a situation in August 2020.”  Was there another example in 2021 as well, such as on July 9^th?

Bonneville is seeking clarification as we believe the appropriate trigger to limit the real-time market’s dispatch of additional energy transfers into a BAA when a BAA is under an energy emergency and no longer able to provide its expected energy requirements. As noted in previous comments, a BAA that is under such an energy emergency is energy deficient and should appropriately fail the EIM RSE and incur the associated failure consequences.

However, Bonneville does not believe the actions a BAA takes when in an energy emergency should be the deciding factor for whether or not the BAA should be failing the EIM RSE. A more appropriate and transparent measure of resource insufficiency during an energy emergency is the stage of emergency the BAA is in.

Bonneville believes entities should be failing the EIM RSE when NERC Energy Emergency Alert Stage (EEA) 2 and/or 3 is declared. We reiterate NERC’s definition of the circumstances around EEAs[1]:

• EEA 1: All available generation resources in use.
  • The BA is experiencing conditions where all available generation resources are committed to meet firm load, firm transactions, and reserve commitments, and is concerned about sustaining its required Contingency Reserves.
  • Non-firm wholesale energy sales (other than those that are recallable to meet reserve requirements) have been curtailed.

• EEA 2: Load management procedures in effect
  • The BA is no longer able to provide its expected energy requirements and is an energy deficient BA.
  • An energy deficient BA has implemented its Operating Plan(s) to mitigate emergencies.
  • An energy deficient BA is still able to maintain minimum Contingency Reserve requirements

• EEA 3: Firm load interruption is imminent or in progress
  • The energy deficient BA is unable to meet minimum Contingency Reserve requirements.

The CAISO has provided a comparison of the CAISO AWE levels with NERC EEA levels[2] and has indicated that a CAISO AWE Warning – triggering DR programs and CAISO AWE Stage 1 is equivalent to a NERC EEA 2 and CAISO AWE Stage 2 and 3 are equivalent to a NERC EEA 3. Hence, Bonneville recommends an entity should be failing the EIM RSE and have EIM transfers limited if they are either declaring a NERC EEA 2 or 3 or its equivalent as in the case of the CAISO.

CAISO continuing to report on how the EIM RSE performed during these instances will provide helpful indicators on the accuracy of the EIM RSE. While Bonneville supports limiting of transfers in these circumstances, we believe the test results should strive to accurately indicate an EIM RSE failure in these circumstances without the need for automatic failure.

[1] See NERC Attachment 1-EOP-011-1

[2] P. 20 of 2021 Summer Readiness Update Call May 26 2021

12. Please provide your organization's preference for the start of phase 2 of the resource sufficiency evaluation enhancement initiative, as well as the expected implementation date of changes made in that phase:

Bonneville’s preference is for CAISO to begin Phase 2 of the EIM RSE enhancements initiative immediately after the completion of Phase 1. If the CAISO agrees that the EIM RSE is foundational to the Western EIM, then there should be no delay to make enhancements necessary to ensure the EIM RSE works as intended.

CAISO notes that the first objective of the EIM RSE is to ensure that BAAs do not lean on the real-time capacity, flexibility, and transmission of other BAAs in the EIM footprint. And CAISO stated that preventing leaning is accomplished through an accurate and transparent measure of the capacity and ramping capability made available by each BAA. Certainly accuracy and transparency of the EIM RSE is a critical first step to achieving its purpose of preventing leaning and hence have for the most part been prioritized in Phase 1. But a measurement in and of itself does not prevent leaning. Bonneville emphasizes that the measurement must be combined with effective failure consequences to actually prevent (or strongly discourage) leaning. As such Bonneville urges the CAISO to prioritize Phase 2 of this initiative to begin January 2022. Bonneville also requests the EIM RSE principles[1] be modified to include that the consequences of EIM RSE failures should prevent (or strongly discourage) leaning. Achieving this principle should be a primary focus of Phase 2. 

[1] P. 6 of the EIM Resource Sufficiency Evaluation Draft Final Proposal

13. Please provide your organization's comments on the proposed EIM Governing Body classification to have joint authority to approve the EIM RSE final proposal:

Bonneville agrees with the CAISO’s decision classification that the EIM Governing Body has joint authority with the Board of Governors over the tariff rule changes proposed in Phase 1 based on the change in role of the EIM Governing Body made on September 23, 2021.

14. Please provide any additional comments on the EIM RSE Enhancements initiative that have not previously been addressed:

There are three specific areas of uncertainty that the CAISO is choosing not to address with this Draft Final Proposal for Phase 1 of this EIM RSE Enhancements policy initiative that also contribute to Bonneville’s opposition to this proposal:

• Net load uncertainty
• Imbalance Conformance, and
• Intertie Deviation Uncertainty

Net load uncertainty

The DMM’s recent monthly reporting on EIM RSE metrics demonstrates that there is an uptick in EIM Entity BAA incidences of failure following the CAISO’s implementation of the addition of the net load uncertainty requirement to the BRCT on June 16, 2021:

While Bonneville supported adoption of this stop gap measure as part of the Summer Readiness Policy Initiative, the CAISO also committed to follow-on analysis and structured its tariff such that it could readily disable this new functionality.  Bonneville believes DMM’s initial analysis and the experience of this past summer compels this analysis as part of Phase 1.  Bonneville recognizes that an increase in failures is an expected outcome of adding the uncertainty requirement.  However, what can we learn from summer 2021 that informs stakeholders – including the MSC – that this uncertainty functionality should or should not be carried forward into summer 2022 unchanged from summer 2021?  If changes are warranted, CAISO should commit to those remedial actions that are implementable now or within Phase 1.  If this cannot be accomplished, then Bonneville recommends CAISO disable this new uncertainty requirement until such time that the accuracy of the calculation can be more thoroughly evaluated or a new approach is implemented.

Imbalance Conformance

The DMM’s recent monthly reporting on EIM RSE metrics has also continued to stress observations expressed by DMM previously regarding the magnitude and routine frequency of imbalance conformance in the CAISO BAA relative to much lower magnitudes and limited frequencies in other EIM Entity BAAs:

This metric clearly suggests that CAISO operators believe that the uncertainty they believe they need to be prepared to act upon during net peak hours exceeds the uncertainty measured in the flex test and applied to the BRCT since June 16, 2021, whereas operators in other EIM Entity BAAs do not undertake frequent, high magnitude imbalance conformance.

Bonneville commends earlier, episodic CAISO reporting during the May 21^st MSC meeting on the capacity test performance during the summer 2020 heatwave, and in particular on the solar uncertainty in the CAISO BAA:

Bonneville believes both of these metrics are indicators that uncertainty in the CAISO BAA are under-represented in the BRCT.  Bonneville believes this under-representation should be addressed in Phase 1 and believes the most compelling approach is to add the CAISO BAA’s upward imbalance conformance to its EIM RSE requirement.  Alternatively, Bonneville asks if a different temporary fix could be considered in Phase 1 to adjust the incremental net load uncertainty requirement for the CAISO BAA or any EIM Entity BAA by the positive difference between the BAA’s imbalance conformance in that hour and the incremental uncertainty requirement for that hour?  For example, if the CAISO BAA net load uncertainty requirement were 700 MW and the imbalance conformance for HE 19 in the HASP was 1,375 MW, then 675 MW would be added to the BRCT net load uncertainty requirement for the test for the operating hour ending 19.

Intertie Deviation Uncertainty

The CAISO posted an analysis of the Intertie Deviation Uncertainty (IDU) on October 6 in advance of the stakeholder workshop on the Draft Final Proposal.  Bonneville appreciates the CAISO’s analysis of this element of the EIM RSE and it raises important concerns that should be immediately addressed in this Phase 1 and not deferred for initial consideration in Phase 2 as the CAISO suggested in the workshop.

First, the CAISO analysis indicates that the IDU element of the BRCT contributes to 50% of the overall EIM RSE failures in the April to August period of 2021 and 100% of the failures for many EIM Entities in the peak hours.

Second, the analysis is clear that the IDU element is not functioning as intended for two important reasons.  The analysis raises concerns with the calculation and did not provide details beyond the following:

“ISO identified a software issue that affected the incremental NSI uncertainty requirement calculation used in the capacity test for the 28th through the end of each month. During these days, the capacity test did not include incremental NSI uncertainty requirements, so these days were excluded from this analysis.”

The analysis also questions whether the design of the IDU element is functioning effectively, specifically:

“There were no balancing areas between April 2021 and August 2021, which had the net-schedule uncertainty requirement covering the actual deviation for more than 95% of intervals in all hours.”[1]

This is important since the design is based on a 95% confidence interval of the observations in the last 90 days.

The third reason why the IDU element should be immediately addressed is that the violin plots in the analysis clearly demonstrate that one or very few actual intertie deviations in a particular month are not predictive of a similar intertie deviation in the same hour of a subsequent month, but nonetheless drive the BRCT requirement higher in those subsequent months.  Bonneville agrees with CAISO that Figure 18 illustrates this clearly.  One or very few observations of a large magnitude intertie deviation(s) in HE 18 and 19 in June and July drove the August requirement in HE 18 and 19 higher.  However, there were not any observed intertie deviations in August or in the months prior to June.  These patterns are repeated in the following Figures:

• APS (Figure 15, HE 19 and 20 in July increasing August requirement with limited or no August observations),
• CAISO (Figure 17 (HE 18 and 19 in July less impactful on the August requirement, but no August observations)
• Pacificorp East (Figure 20, HE 19 and 20 in July more than doubling the August requirement with no August observations)
• Pacifcorp West (Figure 21, HE 20, 21, 22 and 23 in June contributed to August requirement and there were no similar observations in July or August), and
• Salt River Project

These observations demonstrate that intertie deviations are inherently unpredictable and that the current design is an ineffective combination of a limited sample size and high confidence level.

Bonneville believes the IDU element should be disabled in Phase 1 in order to rectify calculation errors and determine in Phase 2 whether a re-designed IDU element should significantly reduce the confidence intervals the CAISO suggested, increase the sample size, eliminate outliers or permanently disable the IDU element of the BRCT.  This phased approach will immediately address the significant contribution to EIM Entities’ EIM RSE failures, the lack of coverage meeting the intent of the element, the fundamental issue that an outage or derate on an intertie in a particular hour within the past 90-days is not predictive of a similar outage occurring in the same hour being tested in the current operating day and mitigate an outcome similar to 2021 in the summer of 2022 when even more EIM Entities – including Bonneville beginning in June 2022 – could be significantly affected by the contribution of the IDU to BRCT requirement.

[1] Analysis of the Intertie Deviation Adder Used in the Capacity Test, CAISO, Page 8.

1. Please provide a summary of your organization’s position on the draft final proposal:

Please see the attached PDF below for DMM's full comments.

The comments should also be available within a few business days of 10-22-2021 in the "Comments on stakeholder processes" section of DMM's website at:  http://www.caiso.com/market/Pages/MarketMonitoring/MarketMonitoringReportsPresentations/Default.aspx#comments

2. Please provide your organization's comments on the proposed modifications to the capacity test - intertemporal constraints proposal:

Please see the attached PDF below for DMM's full comments.

3. Please provide your organization's comments on the value of adding additional rules to the capacity test to more accurately count the capacity that while offline during the RSE, was offered into the real-time market for use:

Please see the attached PDF below for DMM's full comments.

4. Please provide your organization's comments on the consideration of storage resources by the RSE:

Please see the attached PDF below for DMM's full comments.

5. Please provide your organization's comments on the proposed modifications to the flexible ramping test proposal:

Please see the attached PDF below for DMM's full comments.

6. Please provide your organization's comments on the proposed modifications to the balancing test proposal:

Please see the attached PDF below for DMM's full comments.

7. Please provide your organization's comments on the clarifications to the demand response proposal:

Please see the attached PDF below for DMM's full comments.

8. Please provide your organization's comments on the reporting and any additional metrics your organization believes may be valuable:

Please see the attached PDF below for DMM's full comments.

9. Please provide your organization’s comments on the level of EIM entity data granularity supporting the Department of Market Monitoring’s analysis that should be made publically available:

Please see the attached PDF below for DMM's full comments.

10. Please provide your organization's comments on the potential to relax the under-scheduling tests requirement or consequences when demand response is used. If your organization see’s value in this approach, please provide additional detail on how you envision such relaxation being structured:

Please see the attached PDF below for DMM's full comments.

11. Please provide your organization's comments on the clarifications to the emergency actions proposal:

Please see the attached PDF below for DMM's full comments.

12. Please provide your organization's preference for the start of phase 2 of the resource sufficiency evaluation enhancement initiative, as well as the expected implementation date of changes made in that phase:

Please see the attached PDF below for DMM's full comments.

13. Please provide your organization's comments on the proposed EIM Governing Body classification to have joint authority to approve the EIM RSE final proposal:

Please see the attached PDF below for DMM's full comments.

14. Please provide any additional comments on the EIM RSE Enhancements initiative that have not previously been addressed:

Please see the attached PDF below for DMM's full comments.

1. Please provide a summary of your organization’s position on the draft final proposal:

Please see attachment. 

2. Please provide your organization's comments on the proposed modifications to the capacity test - intertemporal constraints proposal:

Please see attachment. 

3. Please provide your organization's comments on the value of adding additional rules to the capacity test to more accurately count the capacity that while offline during the RSE, was offered into the real-time market for use:

Please see attachment. 

4. Please provide your organization's comments on the consideration of storage resources by the RSE:

Please see attachment. 

5. Please provide your organization's comments on the proposed modifications to the flexible ramping test proposal:

Please see attachment. 

6. Please provide your organization's comments on the proposed modifications to the balancing test proposal:

Please see attachment. 

7. Please provide your organization's comments on the clarifications to the demand response proposal:

Please see attachment. 

8. Please provide your organization's comments on the reporting and any additional metrics your organization believes may be valuable:

Please see attachment. 

9. Please provide your organization’s comments on the level of EIM entity data granularity supporting the Department of Market Monitoring’s analysis that should be made publically available:

Please see attachment. 

10. Please provide your organization's comments on the potential to relax the under-scheduling tests requirement or consequences when demand response is used. If your organization see’s value in this approach, please provide additional detail on how you envision such relaxation being structured:

Please see attachment. 

11. Please provide your organization's comments on the clarifications to the emergency actions proposal:

Please see attachment. 

12. Please provide your organization's preference for the start of phase 2 of the resource sufficiency evaluation enhancement initiative, as well as the expected implementation date of changes made in that phase:

Please see attachment. 

13. Please provide your organization's comments on the proposed EIM Governing Body classification to have joint authority to approve the EIM RSE final proposal:

Please see attachment. 

14. Please provide any additional comments on the EIM RSE Enhancements initiative that have not previously been addressed:

Please see attachment. 

1. Please provide a summary of your organization’s position on the draft final proposal:

PG&E supports the proposal with caveats.

PG&E appreciates CAISO’s efforts to improve the accuracy and transparency of the Resource Sufficiency Evaluation.  PG&E continues to be concerned, however, that the draft final proposal does not provide for an adequate understanding of whether the RSE will be administered equitably in all EIM areas.  PG&E would like to see the following elements addressed before CAISO finalizes Phase 1 of this initiative:

1. Address asymmetries between CAISO and non-CAISO areas in the administration of the RSE

There are several areas in which the RSE treats the CAISO BAA and non-CAISO BAAs differently.  PG&E appreciates efforts to create equitable RSE requirements across the structural and market differences between the CAISO and non-CAISO EIM areas, but would like more attention given to the following issues:

• The CAISO BAA’s lack of a mechanism to cure RSE deficiencies between the advisory and binding RSE tests
• Need for clarification of different processes for crediting imports and tagging requirements in the RSE
• Different standards for crediting demand response and storage resources in the RSE
• Asymmetry in emergency protocols that could affect limits on EIM transfers

As PG&E has identified previously, we continue to support an independent function in the CAISO to act and advocate on behalf of the California transmission owners within the CAISO control area. In other stakeholder initiatives[1], PG&E has suggested the creation of a new Office of Balancing Area Affairs within the CAISO.  This new Office, paid for by BAA members only and consisting of (at minimum) a dedicated director-level position, would represent and advocate on behalf of the CAISO BAA interest, as distinct from the broader CAISO efforts.  Ideally, the independent function would represent the CAISO BAA on matters such as the RSE and alleviate the tension caused by CAISO’s dual role as the EIM market operator and an EIM participant.

2. Allow time for DMM to publish counterfactual analysis of RSE proposals prior to finalizing this proposal

The recent DMM RSE performance reports over the summer of 2021 provided valuable insight and information to stakeholders. PG&E appreciates DMM plans to provide further counterfactual analysis on this initiative’s proposed changes to the capacity test and flexibility test.[2]  DMM’s analysis represents critical due diligence to vet and understand the proposed changes in this initiative.  PG&E encourages CAISO to review and implement and any appropriate policy adjustments based on this analysis before finalizing any rule changes in this initiative.

3. Support DMM recommendation to update the intertie uncertainty calculation

PG&E supports DMM’s comments[3] to recalculate the intertie uncertainty calculation to only calculate the uncertainty of tagged schedules at T-40 not delivering (i.e., removing untagged schedules from the calculation).  This change would reflect the CAISO’s proposal to discount CAISO BAA imports award without valid transmission profile e-tags and avoid the potential for the RSE to overcount undelivered CAISO imports. 

[1] See PG&E comments on the Western EIM governance review initiative: https://stakeholdercenter.caiso.com/Comments/AllComments/b48a9f8b-a283-406c-bc3f-94e07b947b7f#org-69424833-7b4f-4df5-a0bb-08afa3bea4e7

[2] DMM plans to provide counterfactual analysis of the proposed use of the STUC time horizon in the capacity test and inclusion of power balance constraint infeasibility in the flexible ramp sufficiency test.  Pg. 34 at http://www.caiso.com/Documents/Presentation-on-Resource-Sufficiency-Evaluation-in-the-EIM-for-Jul-Aug-2021-Sep-28-2021.pdf

[3] See DMM comments, top of pg. 4 at https://stakeholdercenter.caiso.com/Comments/AllComments/917FAC39-93EC-4743-9F5B-7BB25804B4E2#org-88737701-e36a-46d0-a90e-8ef96298ed04

2. Please provide your organization's comments on the proposed modifications to the capacity test - intertemporal constraints proposal:

PG&E supports the consideration of the intertemporal constraints in the RSE as outlined in the draft final proposal. 

3. Please provide your organization's comments on the value of adding additional rules to the capacity test to more accurately count the capacity that while offline during the RSE, was offered into the real-time market for use:

PG&E supports CAISO’s proposal for the RSE to consider offline capacity that was made available to the real-time market (within the STUC time horizon), but may be constrained at the time of the RSE because of previous decisions made by the market optimization.

4. Please provide your organization's comments on the consideration of storage resources by the RSE:

PG&E supports CAISO's proposal to credit storage resources in the RSE and asks for clarification of the application of these rules in CAISO and non-CAISO EIM areas

The draft final proposal includes new rules to differentiate the crediting of storage resources in the RSE from conventional resources.  For the capacity test, a storage resource’s state of charge (SOC) at the time of the RSE, plus any additional energy bids to charge, would be credited as available capacity.  For the flexible ramping sufficiency test, a storage resource’s SOC in the reference market interval at T-7.5, plus any bids to charge or discharge in the upcoming hour, would count towards the ramping requirements.   Unlike other resources, the STUC time horizon will not apply to storage resources in the RSE.   

PG&E sees this as an appropriate approach but wonders how the proposed storage rules may apply differently in the CAISO BAA and for non-CAISO EIM participants that do not participate in the Day-Ahead Market or use a similar Non-Generator Resources (NGR) model.  In support of the goal of equitable and transparent administration of the RSE, it may be helpful for CAISO to clarify if the proposed rules may apply differently across the EIM footprint. 

5. Please provide your organization's comments on the proposed modifications to the flexible ramping test proposal:

PG&E seeks additional clarification on the mechanics of the PBC relaxation and the need for the adder to account for PBC relaxations that may already be addressed by out-of-market actions

PG&E continues to have questions about the proposed change to the flexible ramping test.  PG&E understands that the rationale for increasing the flexible ramping requirement by any power balance constraint (PBC) relaxation amount is to ensure the actual ramping capability needed for the next hour is being tested.  CAISO proposes to increase the upward ramping requirement by the power balance constraint relaxation amount, while the decreasing downward requirement by the same amount.

PG&E has the following clarifying questions:

• Is the relaxation amount also applied to the capacity test? If so, why would the PBC relaxation amount only be applied to the flex test?
• PG&E request the CAISO to clarify the amount of requirement before and after the PBC relaxation considering the following example:

Assume the CAISO’s forecast is 50GW and the available supply is 48GW. The PBC is relaxed for 2GW. PG&E understands that the current RSE requirement equals the original forecast of 50GW (plus uncertainty). The DFP seems to propose increasing the requirement to 52GW.  PG&E requests the CAISO to confirm this understanding is correct.

If so, PG&E is concerned that the proposed modification will double count the insufficiency and produce unnecessary failures. This is because the forecast of 50GW already includes 2GW= 50-48GW insufficiency. On the other hand, it would make more sense to reduce the requirement to 48GW, since the PBC relaxation is usually backed up by out-of-market actions, which address the supply insufficiency.

If PG&E’s understanding is correct that PBC relaxations are usually followed by load adjustments and out-of-market actions such as exceptional dispatch to address a shortfall, wouldn’t the combination of these actions alleviate the need for any additional requirement in the flexible ramping test? 

6. Please provide your organization's comments on the proposed modifications to the balancing test proposal:

The balancing test proposal offers a reasonable compromise that PG&E does not oppose

As stated in previous comments, PG&E does not oppose CAISO’s proposal to exclude the CAISO BAA from revenues from under- and over-scheduling tests because the CAISO BAA is not subject to those tests. 

While the CAISO BAA is not subject to the balancing test – because its scheduling practices and participation in the Day-Ahead Market differ from other EIM areas -- it should be noted that the CAISO BAA still incurs costs and operates its day-ahead and real-time markets to fulfill the purpose of the balancing test to accurately balance demand and protect against under- or over-scheduling.  PG&E supports DMM’s monitoring and reporting the allocation of revenue from over/under-scheduling penalties so that this issue can be revisited in the future if needed.

7. Please provide your organization's comments on the clarifications to the demand response proposal:

8. Please provide your organization's comments on the reporting and any additional metrics your organization believes may be valuable:

PG&E requests assurance of access to BAA-level data

PG&E strongly supports the additional reporting and metrics outlined in the proposal, both on a routine cadence and near real-time.  PG&E requests clarification on what level of test performance data transparency will be provided to CAISO LSEs and on what timeframe the CAISO will provide these data.  PG&E understands detailed sensitive data should not be released on public forums like OASIS, however PG&E requests the CAISO consider alternatives to make BAA-level data available privately to California LSEs in some other forum.

9. Please provide your organization’s comments on the level of EIM entity data granularity supporting the Department of Market Monitoring’s analysis that should be made publically available:

PG&E supports the after-the-fact release of interval-level data of the RSE outcomes and the individual test components for all EIM areas.  This should increase RSE transparency and oversight opportunities by market participants without releasing sensitive market information. 

10. Please provide your organization's comments on the potential to relax the under-scheduling tests requirement or consequences when demand response is used. If your organization see’s value in this approach, please provide additional detail on how you envision such relaxation being structured:

11. Please provide your organization's comments on the clarifications to the emergency actions proposal:

This portion of the proposal belongs in Phase 2 as part of the review of RSE failure consequences

Because this change relates to RSE failure consequences, it may be appropriate to consider this issue as part of the failure consequences review scheduled for Phase 2 of this initiative.  PG&E has previously expressed concern[1] that the current failure consequences could exacerbate reliability challenges and undermine market efficiency.  PG&E supports CAISO’s stated principle that the RSE consequences should not cause operational or reliability issues and looks forward to a further review of how RSE failure consequences may be reformed in Phase 2.

[1] See PG&E comments at https://stakeholdercenter.caiso.com/Comments/AllComments/bbfb322e-1e77-43f0-9256-398928772300#org-2ce704cb-8a12-4ada-ae4a-ef9a140b2339

12. Please provide your organization's preference for the start of phase 2 of the resource sufficiency evaluation enhancement initiative, as well as the expected implementation date of changes made in that phase:

PG&E supports starting Phase 2 after Phase 1 has been implemented and sufficiently reviewed.  Some of the Phase 2 proposals depend on the performance of the changes expected to be implemented in Phase 1.  Starting Phase 2 prematurely may make it difficult to fully understand where problems remain and any appropriate market design solutions. 

13. Please provide your organization's comments on the proposed EIM Governing Body classification to have joint authority to approve the EIM RSE final proposal:

PG&E supports the joint authority classification outlined in the draft final proposal.

14. Please provide any additional comments on the EIM RSE Enhancements initiative that have not previously been addressed:

Clarification of the RSE’s counting of imports and tagging requirements in all EIM areas

PG&E asks for more information on the proposal to discount any CAISO BAA import awards that have not submitted a transmission profile e-Tag equal to their hour ahead scheduling process award by the forty minutes prior to the operating hour (T-40) deadline. 

This proposal to reduce RSE import credits applies specifically to the CAISO BAA, and PG&E would appreciate clarification of how this proposal compares to the RSE standards applied to other EIM BAAs.

Support DMM recommendation to update the intertie deviation uncertainty requirement

PG&E supports DMM’s comments[1] to recalculate the intertie uncertainty calculation to only calculate the uncertainty of tagged schedules at T-40 not delivering (i.e. removing untagged schedules from the calculation).  This change would reflect the CAISO’s proposal to discount CAISO BAA imports award without valid transmission profile e-tags and avoid the potential for the RSE to overcount undelivered CAISO imports. 

[1] See DMM comments, top of pg. 4 at https://stakeholdercenter.caiso.com/Comments/AllComments/917FAC39-93EC-4743-9F5B-7BB25804B4E2#org-88737701-e36a-46d0-a90e-8ef96298ed04

“Clarify that intertie uncertainty calculation will be updated given new tagging requirements

The ISO proposes only counting imports that have been tagged at T-40 in the resource sufficiency tests. Currently the ISO uses historical non-delivery to add uncertainty of import delivery to the tests. This includes imports not tagged by T-40. The ISO should clarify if the intertie uncertainty calculation will be updated to only calculate the uncertainty of tagged schedules at T-40 not delivering.”

1. Please provide a summary of your organization’s position on the draft final proposal:

2. Please provide your organization's comments on the proposed modifications to the capacity test - intertemporal constraints proposal:

3. Please provide your organization's comments on the value of adding additional rules to the capacity test to more accurately count the capacity that while offline during the RSE, was offered into the real-time market for use:

4. Please provide your organization's comments on the consideration of storage resources by the RSE:

PNM believes that storage resources should be considered by the RSE and in general agree this should be examined further in phase 2.  It is important to note thought many EIM entities are procuring both PV and Energy storage together as paired/co-located resources and more consideration is needed due to contractual/tax credit limitations being applied to these sites that require charging to be limited to when the PV is producing energy.  This limits how storage charging can be bid into the market.  This may impact bidding abilities of these resources and therefore impact ability to be part of RSE tests such as bid capacity and flex ramp when they can provide a lot of capability but not able to due to limitations in current CAISO rules.

5. Please provide your organization's comments on the proposed modifications to the flexible ramping test proposal:

6. Please provide your organization's comments on the proposed modifications to the balancing test proposal:

7. Please provide your organization's comments on the clarifications to the demand response proposal:

8. Please provide your organization's comments on the reporting and any additional metrics your organization believes may be valuable:

9. Please provide your organization’s comments on the level of EIM entity data granularity supporting the Department of Market Monitoring’s analysis that should be made publically available:

10. Please provide your organization's comments on the potential to relax the under-scheduling tests requirement or consequences when demand response is used. If your organization see’s value in this approach, please provide additional detail on how you envision such relaxation being structured:

11. Please provide your organization's comments on the clarifications to the emergency actions proposal:

12. Please provide your organization's preference for the start of phase 2 of the resource sufficiency evaluation enhancement initiative, as well as the expected implementation date of changes made in that phase:

13. Please provide your organization's comments on the proposed EIM Governing Body classification to have joint authority to approve the EIM RSE final proposal:

14. Please provide any additional comments on the EIM RSE Enhancements initiative that have not previously been addressed:

PNM believes that the NSI uncertainty/Intertie Deviation Adder should not be included in the Capacity Bid test requirements for the following reasons and respectively request that it be removed as soon as possible and more thoroughly discussed in the stakeholder process as part of a phase 2 item.

• NSI/Intertie transactions are nothing more than the equivalent of Non-Participating Resources (generation for imports and load/non-conforming loads/DDRs for exports).  Entities are not asked to have additional uncertainty if those resources change between the base schedules and either FMM, RTD, actual.  They are accounted for via imbalance charges.  Why should intertie transactions be handled any differently. 
• NSI uncertainty/Intertie Deviation Adder negatively impacts Participating Resources ability to participate in the market that they may have no direct control over. Because each entity has its own OATT/Tariff and must abide by their tariff rules and NERC tagging standards there is nothing other than the imbalance charges that can be applied fairly to all late tagging/intertie scheduling changes.
• The EIM is comprised of a large portion of the Western Interconnect.  Most intertie transactions are with other EIM entities in some form or fashion.  With that in mind you are negatively impact two EIM entities (one in import and one in export) in most cases with this uncertainty.
• Most late tagging/intertie changes are due to last minute changes of participants trying to meet load either due to loss of generation or loss of transmission to deliver energy.  Since imports and exports are really tied to some level of generation in the EIM market this uncertainty is really not realistic to calculate.  Past actions are no proof of future probability so applying on an individual hourly basis based off a histogram for 3 previous months is not an effective way to address this issue.
• Some entities have special scenarios that do not properly account for wheel through energy (DC ties with Eastern Interconnect or ERCOT).  Only half of those intertie wheel throughs are tied to the intertie transactions (Base ETSRs or Tie IDs).  The other half is tied to a NGR due to modeling constraints.  If a late wheel through tag comes in the intertie adder could be due to something because CAISO only sees ½ as intertie when in reality that late tag has not direct impact on the EIM Entity. 
• Another special exception includes intertie transactions tied directly to the output of a Joint Owned Unit where one entity is an EIM Entity, and another is a non-EIM entity.  The two together net each other out, but the EIM entity could be adversely impacted when changes occur after T-40 as the intertie amount changes during the hour and therefore be included in the NSI uncertainty.
• Lastly, there is a pretty high level of administrative burden on both the EIM Entity and CAISO to exclude outlier data.   The entity needs to review each day for hours that may need to be excluded, investigate why there was a deviation and then submit a request to CAISO to review.   This is not a sustainable practice and why PNM believes temporarily removing the uncertainty until a better approach and process can be developed should be considered.

1. Please provide a summary of your organization’s position on the draft final proposal:

Please see Powerex's comments at https://powerex.com/sites/default/files/2021-10/Powerex%20comments%20on%20EIM%20RSE%20draft%20final_0.pdf 

2. Please provide your organization's comments on the proposed modifications to the capacity test - intertemporal constraints proposal:

Please see Powerex's comments at https://powerex.com/sites/default/files/2021-10/Powerex%20comments%20on%20EIM%20RSE%20draft%20final_0.pdf 

3. Please provide your organization's comments on the value of adding additional rules to the capacity test to more accurately count the capacity that while offline during the RSE, was offered into the real-time market for use:

Please see Powerex's comments at https://powerex.com/sites/default/files/2021-10/Powerex%20comments%20on%20EIM%20RSE%20draft%20final_0.pdf 

4. Please provide your organization's comments on the consideration of storage resources by the RSE:

5. Please provide your organization's comments on the proposed modifications to the flexible ramping test proposal:

Please see Powerex's comments at https://powerex.com/sites/default/files/2021-10/Powerex%20comments%20on%20EIM%20RSE%20draft%20final_0.pdf 

6. Please provide your organization's comments on the proposed modifications to the balancing test proposal:

Please see Powerex's comments at https://powerex.com/sites/default/files/2021-10/Powerex%20comments%20on%20EIM%20RSE%20draft%20final_0.pdf 

7. Please provide your organization's comments on the clarifications to the demand response proposal:

8. Please provide your organization's comments on the reporting and any additional metrics your organization believes may be valuable:

Please see Powerex's comments at https://powerex.com/sites/default/files/2021-10/Powerex%20comments%20on%20EIM%20RSE%20draft%20final_0.pdf 

9. Please provide your organization’s comments on the level of EIM entity data granularity supporting the Department of Market Monitoring’s analysis that should be made publically available:

Please see Powerex's comments at https://powerex.com/sites/default/files/2021-10/Powerex%20comments%20on%20EIM%20RSE%20draft%20final_0.pdf 

10. Please provide your organization's comments on the potential to relax the under-scheduling tests requirement or consequences when demand response is used. If your organization see’s value in this approach, please provide additional detail on how you envision such relaxation being structured:

Please see Powerex's comments at https://powerex.com/sites/default/files/2021-10/Powerex%20comments%20on%20EIM%20RSE%20draft%20final_0.pdf 

11. Please provide your organization's comments on the clarifications to the emergency actions proposal:

Please see Powerex's comments at https://powerex.com/sites/default/files/2021-10/Powerex%20comments%20on%20EIM%20RSE%20draft%20final_0.pdf 

12. Please provide your organization's preference for the start of phase 2 of the resource sufficiency evaluation enhancement initiative, as well as the expected implementation date of changes made in that phase:

Please see Powerex's comments at https://powerex.com/sites/default/files/2021-10/Powerex%20comments%20on%20EIM%20RSE%20draft%20final_0.pdf 

13. Please provide your organization's comments on the proposed EIM Governing Body classification to have joint authority to approve the EIM RSE final proposal:

14. Please provide any additional comments on the EIM RSE Enhancements initiative that have not previously been addressed:

Please see Powerex's comments at https://powerex.com/sites/default/files/2021-10/Powerex%20comments%20on%20EIM%20RSE%20draft%20final_0.pdf 

1. Please provide a summary of your organization’s position on the draft final proposal:

The Public Generating Pool (PGP[1]) appreciates the opportunity to comment on the EIM RSE Enhancements Issue Paper and Workshop.  PGP takes great interest in this initiative as there are PGP members who are (or will be) EIM participants that will be directly impacted by the outcome of this initiative.  

PGP is appreciative of the effort made by CAISO and stakeholders to develop a proposal for EIM Resource Sufficiency Enhancements (RSE) that will result in meaningful improvements to the accuracy of the RSE tests.  It is important to acknowledge that there is broad stakeholder support for a number of these improvements, including:

• Requiring a transmission profile at T-40
• Freezing EIM transfers when a market participant is in emergency conditions
• Accounting for any quantity of power balance relaxation in the flexible ramping test
• Excluding CAISO from receiving revenues from imbalance penalties
• Allowing EIM Entities to apply demand response as a resource
• Routine reporting of RSE metrics by DMM

While PGP is supportive of these improvements, there are a few caveats that should be addressed before PGP can fully support this proposal.  First, PGP continues to believe that it is appropriate that CAISO apply the balancing test to itself and any other entities that do not submit base schedules.  Secondly, PGP is concerned that utilizing the STUC time horizon in the capacity test is inconsistent with the principle that the RSE “should accurately and transparently measure the capacity and ramping capability of a balancing authority area prior to allowing additional incremental transfers into or out of the balancing authority”.  Finally, PGP remains concerned about the existing frequency and magnitude of operator load adjustments in CAISO.  These caveats are elaborated on later in these comments.

While the first two caveats are important in theory, it is not clear how important they are in practice.  As such, PGP requests that DMM specifically include routine reporting of:

• The frequency and magnitude of the amount that CAISO or any entity that does not submit base schedules would have failed the balancing test if it were applied to them.
• The frequency and magnitude of resources’ capacity that were counted in the capacity test in the STUC time horizon that were not available to be dispatched at T-2 along with the RSE capacity headroom that existed T-2.

This routine reporting will allow stakeholders to determine whether or not these caveats become important enough in practice to warrant revisiting.

[1] PGP represents eleven consumer-owned utilities in Washington and Oregon that own almost 8,000 MW of generation, approximately 7,000 MW of which is hydro and over 97% of which is carbon free. Four of the PGP members operate their own balancing authority areas (BAAs), while the remaining members have service territories within the Bonneville Power Administration’s (BPA) BAA. As a group, PGP members also purchase over 45 percent of BPA’s preference power.

2. Please provide your organization's comments on the proposed modifications to the capacity test - intertemporal constraints proposal:

CAISO has indicated that it agrees with stakeholders that capacity from resources that are unavailable because of start-up or cycling time should not be counted in the capacity test.  However, CAISO believes that excluding resources that are unavailable due to market optimization would create adverse market incentives.  To strike a balance, in both the straw and draft final proposals, CAISO proposes to count offline capacity that has a start-up time no longer than the period of time covered by the short-term unit commitment (STUC) horizon (approximately 4.5 hours) unless this offline capacity was not following a binding market instruction.

PGP agrees with the principle that the RSE should accurately measure the achievable capacity of the BAA, and PGP continues to believe that this proposal falls short in this area.  CAISO has acknowledged this point, but believes that applying a shorter time horizon “would undermine the market’s efficiency and could create adverse market incentives”.  CAISO cites an example of a resource staying online during the STUC process that would have otherwise been available for economic displacement.  PGP agrees that this could occur with a shorter time horizon, but believes that this is a reasonable tradeoff for having an accurate measurement of a BAA’s achievable capacity. 

Given that there are two competing principles - market efficiency and accurate capacity measurement - PGP believes that the best path forward is for DMM to construct regular reporting of resources that were counted in the RSE capacity test but were not online at T-2 due to being economically displaced in the STUC process as well as the RSE capacity headroom at T-2.  This will provide stakeholders with meaningful data on how often this occurs and the size of these resources that are not available during stressed system conditions.  PGP would like CAISO to commit to reevaluate this issue if stakeholders become concerned that the unavailability of these resources is having a detrimental impact on the market.

3. Please provide your organization's comments on the value of adding additional rules to the capacity test to more accurately count the capacity that while offline during the RSE, was offered into the real-time market for use:

PGP has no comment on the additional rules for resource start-up time.

4. Please provide your organization's comments on the consideration of storage resources by the RSE:

PGP would like additional clarification on the application of rules for storage resources on hydro resources.  CAISO states that “storage resources are different from conventional resources as they have limited continuous energy production”.  Hydro resources frequently experience conditions in which there is limited continuous energy production – especially during periods of low river flow that occur in summer.  Is it CAISO’s intention to apply these storage rules to any resources that may have limited energy production?  If so, PGP would like CAISO to provide additional information on how these storage rules may be applicable to hydro resources.  If not, PGP would like CAISO to specify exactly which resources would be subject to the storage rules

5. Please provide your organization's comments on the proposed modifications to the flexible ramping test proposal:

PGP supports CAISO’s proposal to increase the accuracy of the flexible ramping test to account for any quantity of power balance relaxation that may be present in the market solution.  PGP agrees that this change will ensure that the reference point in the flexible ramping test has a more accurate ramping requirement due to capacity shortfalls.   

6. Please provide your organization's comments on the proposed modifications to the balancing test proposal:

In the draft final proposal, CAISO acknowledges that there is a potential for CAISO to have unbalanced schedules when the power balance constraint is relaxed due to a lack of available supply.  However, CAISO suggests that market participants that do not submit base schedules (such as CAISO) should not be subjected to imbalance charges since there is no mechanism that would allow these participants to intentionally over- or under-schedule. CAISO provided some helpful examples of how entities that submit base schedules could financially benefit from intentionally over- or under-scheduling.  CAISO also reiterated that the intent of the balancing test is to discourage market participants from this type of behavior.  

PGP continues to believe that it is appropriate that CAISO apply the balancing test to itself and any other entities that do not submit base schedules as a way to demonstrate that all elements of the RSE are applied equitably to all market participants – especially since it is not clear whether or not there are any implementation challenges with applying the balancing test to all market participants.  However, PGP believes that a positive step forward is for DMM to include instances when CAISO has unbalanced schedules as a result of lack of available supply.  This will provide stakeholders with meaningful data on how often this occurs and the size these imbalances.  PGP would like CAISO to commit to reevaluate this issue if stakeholders become concerned that CAISO unbalanced schedules are having a detrimental impact on the market. 

7. Please provide your organization's comments on the clarifications to the demand response proposal:

PGP has no comment on the clarifications to the demand response proposal.

8. Please provide your organization's comments on the reporting and any additional metrics your organization believes may be valuable:

PGP has no comment on reporting and additional metrics. 

9. Please provide your organization’s comments on the level of EIM entity data granularity supporting the Department of Market Monitoring’s analysis that should be made publically available:

PGP has no comment on the data that should be made publicly available. 

10. Please provide your organization's comments on the potential to relax the under-scheduling tests requirement or consequences when demand response is used. If your organization see’s value in this approach, please provide additional detail on how you envision such relaxation being structured:

PGP has no comment on relaxing the under-scheduling test requirements when DR is used.

11. Please provide your organization's comments on the clarifications to the emergency actions proposal:

In comments to the straw proposal, PGP supported “limiting EIM transfers when an EEA is in effect, irrespective of whether load has been armed for reserves”.  CAISO has indicated their desire to not link actions of the market operator with the actions of the Reliability Coordinator.  PGP agrees that keeping these functions independent is important and supports the proposal to limit EIM transfers when load is being armed for reserves.

12. Please provide your organization's preference for the start of phase 2 of the resource sufficiency evaluation enhancement initiative, as well as the expected implementation date of changes made in that phase:

PGP believes it is appropriate to begin Phase 2 of the RSE enhancements initiative immediately after the completion of Phase 1. 

13. Please provide your organization's comments on the proposed EIM Governing Body classification to have joint authority to approve the EIM RSE final proposal:

PGP agrees that it is appropriate for the EIM Governing Body to have joint authority on this initiative.

14. Please provide any additional comments on the EIM RSE Enhancements initiative that have not previously been addressed:

1. In comments on the straw proposal, PGP stated that achieving the correct P95 confidence interval requires more data that looking back 90 days.  Since then, CAISO has published analysis (http://www.caiso.com/InitiativeDocuments/Analysis-IntertieDeviationAdderUsed-CapacityTest.pdf) on the impact of the intertie deviation adder now used in the capacity test.  One of the findings of this analysis was that some EIM Entities had very small deviations on most hours, but significant deviations a few of the hours.  It is PGP’s understanding that these large deviations were driven by very rare events, such as the loss of a major transmission line.  The result of this is that an EIM Entity sees volatile uncertainty requirements which suggests that the methodology for establishing these uncertainty requirements is not truly capturing a P95 confidence interval.  PGP supports CAISO’s proposal to revise the intertie uncertainty in Phase 2 of this initiative.

2. CAISO has stated that improvements in market design are the best way to address the need for CAISO operators to address systemic load forecast adjustments and proposes to defer addressing RSE implications to Phase 2 of this initiative.  Two in-flight initiatives (day-ahead market enhancements and flexible ramping product) are intended to have features that should reduce the need for operators for make load adjustments.  CAISO proposes to revisit this issue in Phase 2 to see if these adjustments are still systemic after implementation of these two in-flight initiatives.

In the draft final proposal, CAISO further elaborates that after these in-flight initiatives are completed…

CAISO operators will likely continue to adjust upward, the load forecast the RTPD uses in the event they deem committing additional internal generation or obtaining additional import energy is appropriate to ensure reliability. These load forecast adjustments achieve a similar result to what EIM entities can achieve through their base scheduling practice and through bilaterally transacting for imports. Neither of these EIM entity actions are incorporated into the demand forecast used by the RSE. 

PGP acknowledges that there are rare instances when an EIM entity will secure bilateral purchases when there is a concern that actual demand may be higher than forecasted demand and that the EIM Entity is not required to adjust their load forecast.  

CAISO goes on to state that…

By considering load forecast adjustments as part of phase 2 of the initiative, the ISO and stakeholders will have the opportunity to evaluate the frequency of load forecast adjustments after the planned market enhancements to determine if developing a methodology to account for, and add to the requirements of the RSE, how all EIM entities procure capacity that may be in excess of their forecasted demand prior to EIM participation is appropriate. 

Given that DMM has begun to provide additional information on these load adjustments that CAISO makes, PGP agrees that this data will provide an opportunity to evaluate the frequency and magnitude of these load adjustments after the in-flight initiatives are completed.  

However, PGP continues to be concerned that the frequency and magnitude these load adjustments that currently exists in CAISO relative to the EIM Entities indicates that this issue warrants attention sooner rather than later.  The DMM report for September summarizes this point in the following graphs. Because this difference is so striking, PGP believes this should be addressed in Phase 1 of this initiative.

1. Please provide a summary of your organization’s position on the draft final proposal:

See attached document

2. Please provide your organization's comments on the proposed modifications to the capacity test - intertemporal constraints proposal:

See attached document

3. Please provide your organization's comments on the value of adding additional rules to the capacity test to more accurately count the capacity that while offline during the RSE, was offered into the real-time market for use:

See attached document

4. Please provide your organization's comments on the consideration of storage resources by the RSE:

See attached document

5. Please provide your organization's comments on the proposed modifications to the flexible ramping test proposal:

See attached document

6. Please provide your organization's comments on the proposed modifications to the balancing test proposal:

See attached document

7. Please provide your organization's comments on the clarifications to the demand response proposal:

See attached document

8. Please provide your organization's comments on the reporting and any additional metrics your organization believes may be valuable:

See attached document

9. Please provide your organization’s comments on the level of EIM entity data granularity supporting the Department of Market Monitoring’s analysis that should be made publically available:

See attached document

10. Please provide your organization's comments on the potential to relax the under-scheduling tests requirement or consequences when demand response is used. If your organization see’s value in this approach, please provide additional detail on how you envision such relaxation being structured:

See attached document

11. Please provide your organization's comments on the clarifications to the emergency actions proposal:

See attached document

12. Please provide your organization's preference for the start of phase 2 of the resource sufficiency evaluation enhancement initiative, as well as the expected implementation date of changes made in that phase:

See attached document

13. Please provide your organization's comments on the proposed EIM Governing Body classification to have joint authority to approve the EIM RSE final proposal:

See attached document

14. Please provide any additional comments on the EIM RSE Enhancements initiative that have not previously been addressed:

See attached document

1. Please provide a summary of your organization’s position on the draft final proposal:

Salt River Project Agricultural Improvement and Power District (SRP) supports the CAISO’s efforts to improve the accuracy and transparency of the EIM Resource Sufficiency Evaluation (RSE) and believes the Draft Final Proposal includes enhancements that will help achieve that objective.

SRP continues to have concerns related to the details of applying intertemporal constraints to the capacity test and appreciates this opportunity to comment. As explained in item 2 below, SRP is specifically concerned about the hourly and real-time pre-dispatch (RTPD) load forecast that is used to calculate the net uncertainty requirement for the flex ramping sufficiency (flex) and capacity tests. SRP also seeks further clarification on proposed modifications to the flex test.

In addition to these comments, SRP also supports the joint EIM Entities comments.

2. Please provide your organization's comments on the proposed modifications to the capacity test - intertemporal constraints proposal:

SRP generally supports the proposal for the intertemporal constraint modifications; however, we have concerns related to the dialogue during the October 12 stakeholder meeting that referenced the capacity test only accounting for PMin or an average capacity. Because the capacity test is performed for each 15-minute interval, a resource can have varying capacity available during the hour. Accounting for a more stringent capacity test would put a burden on the EIM entities to have additional capacity available for the test that is not really needed in real time, thus providing a less optimized dispatch.

SRP appreciates the capacity test examples the CAISO provided in the proposal but requests further clarification regarding how the resource was base scheduled for the RSE test in the examples. SRP understood that the resource under evaluation was base scheduled at zero for the RSE test, regardless of its status, to account up to 400 MW of capacity where applicable. Is SRP’s assumption that the resource was base scheduled at zero correct?

SRP still has concerns regarding the hourly and real-time pre-dispatch (RTPD) load forecast that CAISO uses to calculate the net uncertainty requirement for the flex and capacity tests. SRP has experienced an increase in failures of the capacity test, and the primary reason for this increase was the additional uncertainty requirement from the flex test. The Department of Market Monitoring (DMM) EIM Resource Sufficiency Evaluation Metrics Report covering September 2021 shows that without this additional requirement SRP would have passed the capacity test with good margins. SRP’s concerns focus on cases when we fail the capacity test due to the uncertainty requirement for the interval but pass the flex ramp test or have no infeasible solution indicating lack of capacity during that hour. While the quantile regression method may help improve calculation of the uncertainty requirement, we understand it may not be implemented in time for next summer. This puts SRP in a position that will hinder participation in EIM due to failing capacity tests because of the inaccurate hourly and RTPD load forecast. As noted in CIDI tickets SRP has submitted to the CAISO, SRP seeks and continues to request a solution to this issue sooner than the anticipated implementation of the quantile regression method.

SRP seeks clarification on how capacity accounting will happen during transitions. The CAISO provided a brief explanation at the October 12 stakeholder meeting and stated that when the market decommits the resource from the higher configuration to the lower, the capacity test will account for the higher configuration capacity during the test. The current design of the capacity test does not account for where the resource is operating in real-time; instead, the capacity test accounts for where the base schedules are and evaluates the headroom to Pmax. A base schedule can remain in a different configuration regardless of the market commitment depending on the individual Balancing Authority optimized resource plan. SRP requests additional information on how capacity accounting during transitions will be designed and implemented.

3. Please provide your organization's comments on the value of adding additional rules to the capacity test to more accurately count the capacity that while offline during the RSE, was offered into the real-time market for use:

SRP supports the CAISO’s proposal to include the capacity that was made available through the Short-Term Unit Commitment (STUC) horizon for the RSE test even if the resource was offline during that hour.

4. Please provide your organization's comments on the consideration of storage resources by the RSE:

No comment at this time.

5. Please provide your organization's comments on the proposed modifications to the flexible ramping test proposal:

SRP supports modification to the flex test to include any power balance constraint as an added requirement but seeks clarification. The CAISO recently added the uncertainty requirement from the flex test to the capacity test. SRP seeks clarification on how this additional uncertainty requirement component will affect the capacity test requirement.

SRP also seeks clarification on how flex ramping capability will be accounted for during the transition period or forbidden operating range. If there are two different ramp rates between the configuration, how will the CAISO determine the ramping capability?

6. Please provide your organization's comments on the proposed modifications to the balancing test proposal:

SRP supports the proposed modifications to the balancing test.

7. Please provide your organization's comments on the clarifications to the demand response proposal:

SRP supports the CAISO proposal that provides different ways to include demand response programs. SRP requests that the CAISO also include a mechanism to have the demand response program with a megawatt quantity instead of only allowing the load distribution factor methodology.

8. Please provide your organization's comments on the reporting and any additional metrics your organization believes may be valuable:

Consistent with the joint EIM Entities comments, SRP requests that the CAISO make data for each Balancing Authority Area (BAA) publicly available to address issues in the load forecast and variable energy resource forecasts used in the RSE, which would help solve SRP’s concern described in item 2 above.

9. Please provide your organization’s comments on the level of EIM entity data granularity supporting the Department of Market Monitoring’s analysis that should be made publically available:

SRP would like to see the publicly available data granularity supporting the DMM’s analysis at the BAA level, not the resource level. SRP requests data granularity to the resource level for the quarterly reporting of BAA EIM performance.

10. Please provide your organization's comments on the potential to relax the under-scheduling tests requirement or consequences when demand response is used. If your organization see’s value in this approach, please provide additional detail on how you envision such relaxation being structured:

While this will default to failing the balancing test and application of over/under scheduling test penalties, SRP believes that the 5% under scheduling trigger is still relevant. If it is found after-the-fact that there is a misuse of this ability, SRP could support a tighter trigger than the 5% for under scheduling and an associated increase in penalties.

11. Please provide your organization's comments on the clarifications to the emergency actions proposal:

SRP supports the clarifications to the emergency actions proposal. Even though the current proposal puts the burden on EIM Entities’ operations to communicate such emergencies, it is a step in the right direction.

12. Please provide your organization's preference for the start of phase 2 of the resource sufficiency evaluation enhancement initiative, as well as the expected implementation date of changes made in that phase:

SRP encourages the CAISO to begin Phase 2 immediately after Phase 1.

13. Please provide your organization's comments on the proposed EIM Governing Body classification to have joint authority to approve the EIM RSE final proposal:

SRP agrees that the EIM Governing Body and Board of Governors should have joint authority over this initiative. 

14. Please provide any additional comments on the EIM RSE Enhancements initiative that have not previously been addressed:

As described in the Joint EIM Entities comments, upward load biasing by the CAISO BAA appears to be a significant driver of RSE inaccuracies. SRP encourages the CAISO to address this issue by committing to add the CAISO’s upward load bias used in the fifteen-minute market to its RSE requirements.

SRP requests that the CAISO maintain the original implementation timeline for Phase 1 of no later than the summer 2022.  SRP is concerned that the CAISO will deprioritize important initiatives such as the EIM RSE Enhancements due to the CAISO’s renewed focus on an Extended Day Ahead Market (EDAM). SRP shared a similar concern in our External Load Forward Scheduling Rights Process initiative issue paper comments.

1. Please provide a summary of your organization’s position on the draft final proposal:

2. Please provide your organization's comments on the proposed modifications to the capacity test - intertemporal constraints proposal:

 The Six Cities support this element of the CAISO’s proposal, consistent with their prior comments. 

3. Please provide your organization's comments on the value of adding additional rules to the capacity test to more accurately count the capacity that while offline during the RSE, was offered into the real-time market for use:

 The Six Cities are unclear if this question within the comment template is targeted at specific elements within the proposal on intertemporal constraints.  As the Six Cities have stated in prior comments, the RSE capacity test should count as available resources that were offered into the real time market, but were not dispatched due to prior economic decisions.  The Six Cities also find reasonable the CAISO’s proposals to (1) omit from consideration in the RSE capacity test resources that are offline and participated in the real time market in the RTPD or previously in the STUC horizon, were dispatched, and subsequently did not follow dispatch instructions; and (2) to include in the RSE capacity test resources that are offline as a result of having received and followed dispatch instructions to shut or transition down. 

4. Please provide your organization's comments on the consideration of storage resources by the RSE:

 The CAISO’s proposal with respect to the consideration of storage resources in the RSE – namely, to limit counting within the RSE capacity test of storage resources to the amount of capacity corresponding to the amount of charge at the time of the RSE, plus any additional amount made available through energy bids to charge – appears to be reasonable. 

5. Please provide your organization's comments on the proposed modifications to the flexible ramping test proposal:

 The Six Cities do not have comments on this element of the CAISO’s proposal at this time. 

6. Please provide your organization's comments on the proposed modifications to the balancing test proposal:

 The Six Cities concur with the proposal to continue not applying the balancing test to the CAISO and do not oppose the proposed revision to the allocation of revenue resulting from test failures. 

7. Please provide your organization's comments on the clarifications to the demand response proposal:

 The Six Cities do not oppose the CAISO’s proposed clarifications to the demand response proposal. 

8. Please provide your organization's comments on the reporting and any additional metrics your organization believes may be valuable:

  The Six Cities do not have comments on this element of the CAISO’s proposal at this time. 

9. Please provide your organization’s comments on the level of EIM entity data granularity supporting the Department of Market Monitoring’s analysis that should be made publically available:

  The Six Cities do not have comments on this element of the CAISO’s proposal at this time. 

10. Please provide your organization's comments on the potential to relax the under-scheduling tests requirement or consequences when demand response is used. If your organization see’s value in this approach, please provide additional detail on how you envision such relaxation being structured:

  The Six Cities do not have comments on this element of the CAISO’s proposal at this time. 

11. Please provide your organization's comments on the clarifications to the emergency actions proposal:

 As stated in their prior comments, the Six Cities do not oppose the proposal to limit incremental EIM transfers when firm load is armed for spinning/non-spinning reserves.  At the same time, the Six Cities are concerned that  non-CAISO EIM entities may implement measures to mitigate similar emergency conditions that have not been included by the CAISO as among the conditions that would trigger limitations in incremental EIM transfers.  The Six Cities are concerned that the CAISO area is being targeted for more stringent emergency rules that are inapplicable to other EIM participants.  The Six Cities urge the CAISO to evaluate and include within the scope of this proposal application of EIM transfer limitations during operating conditions when emergency actions are taken that are comparable to the CAISO’s decision to arm load to provide reserves. 

12. Please provide your organization's preference for the start of phase 2 of the resource sufficiency evaluation enhancement initiative, as well as the expected implementation date of changes made in that phase:

 At this time, the Six Cities do not have a position on the timing for phase 2 of this initiative. 

13. Please provide your organization's comments on the proposed EIM Governing Body classification to have joint authority to approve the EIM RSE final proposal:

 The Six Cities concur in the CAISO’s determinations regarding approval authority for this initiative. 

14. Please provide any additional comments on the EIM RSE Enhancements initiative that have not previously been addressed:

 The Six Cities note that the CAISO’s proposal regarding delivery of import schedules does not appear to have been specifically addressed within this comments template.  At this time, the Six Cities do not oppose the CAISO’s proposal at section 5.1.4.2. with respect to the consideration of import schedules that have valid transmission profile e-Tags by T-40.  The Six Cities continue to understand that this requirement is only applicable within the context of establishing the pool of resources eligible to be included in the RSE, and that this requirement does not pertain to or address market participation more generally. 

The Six Cities agree with the CAISO’s determination that the requirement for full e-Tags by T-40 as a precondition to real time market participation would be unreasonable. 

On page 41 of the Draft Final Proposal, the last line appears to include an error – specifically, the explanation of the variables should read “Flexible Ramp Down Requirement” instead of “Flexible Ramp Up Requirement”. 

1. Please provide a summary of your organization’s position on the draft final proposal:

2. Please provide your organization's comments on the proposed modifications to the capacity test - intertemporal constraints proposal:

3. Please provide your organization's comments on the value of adding additional rules to the capacity test to more accurately count the capacity that while offline during the RSE, was offered into the real-time market for use:

4. Please provide your organization's comments on the consideration of storage resources by the RSE:

5. Please provide your organization's comments on the proposed modifications to the flexible ramping test proposal:

6. Please provide your organization's comments on the proposed modifications to the balancing test proposal:

7. Please provide your organization's comments on the clarifications to the demand response proposal:

8. Please provide your organization's comments on the reporting and any additional metrics your organization believes may be valuable:

9. Please provide your organization’s comments on the level of EIM entity data granularity supporting the Department of Market Monitoring’s analysis that should be made publically available:

10. Please provide your organization's comments on the potential to relax the under-scheduling tests requirement or consequences when demand response is used. If your organization see’s value in this approach, please provide additional detail on how you envision such relaxation being structured:

11. Please provide your organization's comments on the clarifications to the emergency actions proposal:

12. Please provide your organization's preference for the start of phase 2 of the resource sufficiency evaluation enhancement initiative, as well as the expected implementation date of changes made in that phase:

13. Please provide your organization's comments on the proposed EIM Governing Body classification to have joint authority to approve the EIM RSE final proposal:

14. Please provide any additional comments on the EIM RSE Enhancements initiative that have not previously been addressed:

1. Please provide a summary of your organization’s position on the draft final proposal:

WPTF appreciates the opportunity to submit these comments on the CAISO’s EIM Resource Sufficiency Evaluation Enhancements Draft Final Proposal discussed with stakeholders during the October 12 stakeholder call. Overall, WPTF supports the CAISO’s efforts to ensure the RSE evaluations are accurate and applied equitably to all BAAs. We understand, and support, the CAISO acknowledging that some of the Phase 1 proposal elements may be considered interim steps due to the need for additional discussions in a subsequent effort or recognizing the need to strike a reasonable balance with complex market designs and potential impact. 

That being said, WPTF is concerned that the CAISO is not moving far enough, fast enough on some key features that are contributing to the inaccuracy of the RSE tests. We strongly urge the CAISO when considering which interim steps to include in Phase 1 that the CAISO prioritize key features that will result in the largest impact to improving the accuracy of the RSE as we head towards Summer of 2022. The analysis of the RSE metrics during the summer 2020 and 2021 events have very similar trends, further supporting the sense of urgency and need to address the issues prior to Summer 2022 to avoid a repeat of events. However, the door is closing rather quickly with what the CAISO can implement prior to Summer 2022. WPTF provides more discussion on this element in response to question #14. We also encourage the CAISO to continue the discussions regarding the treatment of load conformance in the RSE tests.

Additionally, WPTF urges the CAISO to capitalize on the momentum gained through this effort and immediately engage with stakeholders on issues identified to be included in Phase 2 of this initiative. It will be extremely important to also keep these discussions going especially to the extent there is coordination or potential interaction with the issues/market design elements being discussed within the recently re-initiated Extended Day-ahead Market effort.

Lastly, WPTF would like to take this opportunity to reiterate the importance of taking a principled approach to any policy process. WPTF is concerned in this initiative and others, that the CAISO is taking more of a “middle of the road” approach with its proposed changes. In other words, rather than really discussing with stakeholders the goals and principles of each initiative to ensure that the resulting design is aligned with the agreed upon principles, the proposed changes are trying to balance the opinions of the differing parties, potentially at a cost to the overall market efficiency.  

2. Please provide your organization's comments on the proposed modifications to the capacity test - intertemporal constraints proposal:

WPTF is generally supportive of the CAISO’s proposed enhancements to more accurately capture the capacity that has been made available to the real-time market and can be accessed by the market within the evaluation period. However, we do encourage the CAISO to consider additional modifications as it relates to resources offline as discussed in our response to question #3 below.

Additionally, WPTF appreciates the analysis provided whereby the CAISO quantified the amount of capacity from resources within the CAISO BAA that were not available but included in the RSE tests. WPTF asks that the CAISO also provide similar analysis for all the BAAs to provide a more comprehensive picture of the overall impact and size of the issue.

3. Please provide your organization's comments on the value of adding additional rules to the capacity test to more accurately count the capacity that while offline during the RSE, was offered into the real-time market for use:

This is an area where WPTF understands the CAISO is trying to strike a balance between an overly complex test while accurately capturing capacity that is available. Generally speaking, WPTF does find value in exploring additional enhancements that more accurately reflect the amount of capacity available from resources offline during the RSE. We agree with stakeholders that simply crediting the full maximum output of an offline resource that could be started up within the horizon could overstate the capacity. For example, the CAISO could simply consider for those resources either (1) counting only the minimum load capacity or (2) a portion of their total capacity based on an estimated feasible dispatch point.

4. Please provide your organization's comments on the consideration of storage resources by the RSE:

WPTF appreciates the CAISO’s recognition that storage is unique when it comes to determining how much capacity has been made available and thus should be counted in the capacity tests. WPTF seeks additional clarification from the CAISO on how it will calculate the capacity for storage resources. The proposal notes that it will consider the resource’s state of charge and bids to charge when determining the capacity to be used in the RSE and bids to charge/discharge in the Flexible Ramping Test. However, WPTF is unclear if the actual formulation used to determine the capacity for both tests will take into account other parameters such as efficiency losses and/or maximum/minimum stored energy limits. Thus, WPTF asks for additional detail to be provided in the next iteration along with some numerical examples. At that point, WPTF will be in a better position to opine on if additional discussions are needed in Phase 2 of this effort.

5. Please provide your organization's comments on the proposed modifications to the flexible ramping test proposal:

6. Please provide your organization's comments on the proposed modifications to the balancing test proposal:

7. Please provide your organization's comments on the clarifications to the demand response proposal:

8. Please provide your organization's comments on the reporting and any additional metrics your organization believes may be valuable:

9. Please provide your organization’s comments on the level of EIM entity data granularity supporting the Department of Market Monitoring’s analysis that should be made publically available:

WPTF commends the CAISO for proposing to make additional data publicly available to market participants. Transparency is a key element to a well-functioning market and enables both the CAISO and market participants to evaluate the effectiveness of the market using the same data. WPTF encourages the CAISO to post the data in the most granular manner via Oasis that is acceptable given the confidentiality concerns of the data. In other words, we understand some level of aggregation will be needed but we ask that the least amount of aggregation is done such that the value and usefulness of the data to market participants is maximized.

10. Please provide your organization's comments on the potential to relax the under-scheduling tests requirement or consequences when demand response is used. If your organization see’s value in this approach, please provide additional detail on how you envision such relaxation being structured:

11. Please provide your organization's comments on the clarifications to the emergency actions proposal:

12. Please provide your organization's preference for the start of phase 2 of the resource sufficiency evaluation enhancement initiative, as well as the expected implementation date of changes made in that phase:

WPTF supports the CAISO starting Phase 2 of this initiative upon Board decision of Phase 1.

13. Please provide your organization's comments on the proposed EIM Governing Body classification to have joint authority to approve the EIM RSE final proposal:

14. Please provide any additional comments on the EIM RSE Enhancements initiative that have not previously been addressed:

WPTF understands the time constraint that the CAISO is working under to get changes to the RSE in place before Summer 2022. We think everyone can agree that this effort should be aiming to avoid a repeat of the Summer 2020 and 2021 events in 2022. The door is rapidly closing thus we understand the desire and need to phase this initiative. However, we are concerned that the CAISO is deferring consideration of changes to key features to Phase 2 that lend themselves to potential interim solutions that could be considered within Phase 1 – interim solutions that may provide a significant improvement compared to where we are today.

For example, the CAISO’s Intertie Deviation Analysis clearly identifies that there are real concerns with how the current intertie deviation adder is being calculated and applied in the capacity test. However, this element has been deferred to Phase 2 even though based on the CAISO’s own analysis it is clearly contributing to RSE inaccuracies.

Similarly for the net load uncertainty. We appreciate the CAISO including the metrics in the Monthly Summer Market Performance reports that provide an indication of the impact including the net load uncertainty has had on RSE failures. As the CAISO notes, there has been a significant increase in failures since the application of net load uncertainty. Here again is an opportunity for the CAISO to first provide more insights and transparency into what the main drivers may be that is causing the net load uncertainty application to significantly impact the failures. With this additional information, stakeholders could then make a more informed assessment if there are potential interim solutions that will help improve the RSE accuracy through the net load uncertainty application.  

Lastly, throughout these discussions the CAISO continues to point to the Quantile Regression approach as a solution to help address several concerns. If that is in fact a solution for these issues, then WPTF urges the CAISO to prioritize implementing that process sooner rather than later such that the benefits of that modification can be realized prior to summer 2022.

In summary, WPTF urges the CAISO to at a minimum consider interim changes it could make to the intertie deviation adder in Phase 1 that will at least improve the accuracy of the RSE for summer of 2022. WPTF believes there are several potential solutions/modifications that would be a step in the right direction that could be considered in Phase 1, and then allow additional discussions to take place in Phase 2.