A cascading policy learning framework for enhancing power grid resilience

Theoretical foundation of cascading knowledge transfer

The CPLF consists of three main steps that implement progressive knowledge distillation, where each algorithm acts as a teacher for the subsequent one, following the established teacher-student paradigm (Hinton, 2015). This cascading approach goes beyond sequential execution by embedding knowledge transfer as a core design feature. From a transfer learning perspective, the framework cultivates adaptive policies for increasingly dynamic grid scenarios, with the knowledge accumulated in earlier stages informing the learning process of subsequent algorithms.

The mathematical foundation of this cascading approach lends itself to formalization through the lens of transfer learning. Let ${\theta }_1$, ${\theta }_2$, and ${\theta }_3$ represent the parameter sets for PPO, TRPO, and A2C, respectively. The transfer process follows: ${\theta }_2^{(0)}={\theta }_1^{(f)}$ and ${\theta }_3^{(0)}={\theta }_2^{(f)}$ where the superscripts (0) and (f) denote initial and final parameter states. This sequential inheritance allows each stage to leverage the knowledge gained from previous stages, while also applying its distinct optimization constraints. At each stage transition, we transfer the complete actor and critic network parameters (weights and biases) but reinitialize the optimizer state. This design choice allows each algorithm’s distinct optimization dynamics to take full effect while preserving the learned control strategy.

Hence, the three sequential stages are:

1. Initial policy development: Proximal Policy Optimization establishes stable baseline behavior through its clipped surrogate objective function: $$L^{\mathrm{C}\mathrm{L}\mathrm{I}\mathrm{P}}(\theta )={\mathbb{E}}_t\left[min(r_t(\theta ){\hat{A}}_t,\mathrm{c}\mathrm{l}\mathrm{i}\mathrm{p}({\mathrm{r}}_{\mathrm{t}}(\theta ),1-\text{ε},1+\text{ε}){\hat{A}}_t)\right]$$with $\epsilon =0.2$ constraining policy updates. This stability is pivotal for creating a robust initial policy within the highly interconnected power grid environment. The initial PPO stage then acts as a foundational teacher, establishing stable behavioral patterns.

2. Policy fine-tuning: Trust Region Policy Optimization implements mathematically rigorous updates by enforcing Kullback-Leibler (KL) divergence constraints, a statistical measure that quantifies the difference between two probability distributions, building upon PPO’s trained policy network weights: $$\underset{\theta }{max}{\mathbb{E}}_t\left[\frac{{\pi }_{\theta }(a_t\mid s_t)}{{\pi }_{{\theta }_{\mathrm{o}\mathrm{l}\mathrm{d}}}(a_t\mid s_t)}{\hat{A}}_t\right]\mathrm{s}\mathrm{u}\mathrm{b}\mathrm{j}\mathrm{e}\mathrm{c}\mathrm{t}\mathrm{t}\mathrm{o}{D}_{\mathrm{K}\mathrm{L}}({\pi }_{{\theta }_{\mathrm{o}\mathrm{l}\mathrm{d}}},{\pi }_{\theta })\le \delta$$with $\delta =0.01$ sustaining policy stability. This constraint guarantees that the policy variance reduction preserves the useful policy patterns learned in the initial stage while optimizing for unstable grid scenarios, particularly those involving multiple concurrent contingencies or rapid load fluctuations. TRPO then hones these patterns, leveraging its trust region approach to deliver robust policy improvements.

3. Final optimization: Advantage Actor-Critic further optimizes the policy using a learned value function baseline, initialized with TRPO’s trained policy network weights: $$L^{\mathrm{A}2\mathrm{C}}(\theta )={\mathbb{E}}_t\left[\log {\pi }_{\theta }(a_t\mid s_t)(R_t-V(s_t))+\alpha H({\pi }_{\theta })\right]$$with $V(s_t)$ normalizing returns across states. This approach balances exploration and exploitation, normalizing returns across different grid states and resulting in more precise policy improvements, primarily in scenarios with diverse load patterns and grid topologies.

The knowledge transfer between algorithmic stages represents a critical component of the CPLF framework. Each transition involves the inheritance of the complete policy network parameters, where the successor algorithm initializes its neural network weights and biases directly from the trained predecessor model. Specifically, this transfer is intentionally limited to the policy and value function weights and does not include the optimizer’s state (e.g., Adam’s moments). Consequently, each algorithmic stage begins with a re-initialized optimizer. This approach solidifies maximal knowledge retention of the learned control strategy while allowing the distinct optimization dynamics of each algorithm to take full effect without being constrained by the momentum of a previous stage. The transfer process leverages the identical network architectures maintained across all stages, allowing for seamless parameter mapping without dimensional conflicts.

The constraints of each algorithm implicitly regularize the transfer process. PPO’s clipping mechanism ( $\epsilon =0.2$) maintains initial stability. TRPO’s KL divergence constraint ( $\lambda =0.01$) maintains learned behaviors through mathematically rigorous updates, and A2C’s value function baseline empowers precise policy improvements by reducing gradient variance.

Formally, the policy transfer mechanism minimizes the divergence between the source policy ${\pi }_{\mathrm{s}\mathrm{o}\mathrm{u}\mathrm{r}\mathrm{c}\mathrm{e}}$ and target policy ${\pi }_{\mathrm{t}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{t}}$: $\underset{\theta }{min}{D}_{\mathrm{K}\mathrm{L}}({\pi }_{\mathrm{s}\mathrm{o}\mathrm{u}\mathrm{r}\mathrm{c}\mathrm{e}}\Vert {\pi }_{\mathrm{t}\mathrm{a}\mathrm{r}\mathrm{g}\mathrm{e}\mathrm{t}})$, subject to performance constraints that guarantee the target policy maintains or improves upon the performance of the source policy. This theoretical foundation guarantees that our cascading approach optimizes both policy performance and knowledge retention across algorithmic transitions.

The numbered arrows in Fig. 1 illustrate the sequence of operations. Hence, arrows (1, 3, and 5) represent the interaction of each algorithm with the environment. Arrows 2 and 4 represent the transfer of policy network weights between successive algorithms. Finally, arrow 6 depicts the final policy deployment by producing the optimal policy $\pi$.

Algorithm 1 details the proposed framework.

Network architecture and data handling

Our consistent neural network architecture allows for seamless parameter transfer across all algorithmic stages. The network features two fully connected hidden layers with 1,200 and 1,000 units, respectively, utilizing LeakyReLU activation functions with $\alpha =0.01$. This architectural choice delivers sufficient representational capacity for the high-dimensional state space while minimizing resource consumption. Complete neural network parameter transfer occurs between stages, including weights and biases from the predecessor algorithm, with no layer freezing, allowing complete adaptation while preserving learned representations.

Hyperparameter selection followed a systematic manual tuning process informed by values commonly reported in DRL literature. Starting from the defaults of stable-baselines3 (Raffin et al., 2021), we iteratively adjusted the learning rates, batch sizes, and iteration counts based on the agent’s performance on validation scenarios. While this approach yielded stable, high-performing policies, we acknowledge that automated hyperparameter optimization (e.g., Bayesian optimization via Optuna (Akiba et al., 2019)) could further improve performance and represents a valuable direction for future work.

The data for our experiments comes from the Grid2Op environment in a structured format that minimizes preprocessing requirements while maintaining the ontological integrity of power system measurements (Donnot, 2020). Data points are directly integrated into the reinforcement learning agents without requiring transformational procedures such as normalization or feature scaling, thus preserving their ontological integrity. This methodological approach maintains a streamlined operational footprint while mitigating the risk of systematic biases that might otherwise arise through data manipulation techniques.

Experimental setup

Grid2Op (Donnot, 2020) serves as the foundational framework for our resilient grid control experiments. It models the power grid control dynamics as a Markov Decision Process (MDP), which is well-suited for RL. Its compatibility with the OpenAI Gym library (Brockman et al., 2016) further optimizes its flexibility, supporting the development and assessment of a diverse range of control agents, including RL models, heuristics, and optimization algorithms. We employed the Grid2Op Robustness 2020 challenge environment (Marot et al., 2020), which implements a modified subset of the IEEE 118-bus system and uses only traditional energy sources, as depicted in Fig. 2. This environment comprises a total of 36 substations managing power distribution, 59 powerlines for power transmission, 22 generators supplying power, and 37 loads, including interconnections with other grid sections represented as negative loads. Table 2 provides the complete technical specifications of the experimental environment, including the Python version, Grid2Op version, backend details, observation space dimensions, action space size, and attack model parameters. It is worth noting that the detailed attributes of the observation space are presented in Table 3.

Furthermore, this testbed supports a high-dimensional action space comprising over 66,000 discrete and continuous actions, allowing for end-to-end exploration and optimization of power grid performance. It delivers rich observational data to agents, including temporal information such as the month, day, and hour; power flow measurements, e.g., active/reactive power, voltage, and current, at both ends of the power lines; generator and load statistics, including active power output and voltage levels; grid topology status, for instance, bus connections, line status, and capacity; operational constraints, namely cooldown timers for powerlines and substations; and finally maintenance schedules for powerlines.

Agents can interact with this environment through two primary action categories: “Change” operations, which toggle powerline connections and bus assignments, and “Set” operations, which directly assign specific statuses or buses to grid components. The powerline status (connected/disconnected) can be modified either by directly setting its status using “set_line_status” or by assigning components to specific buses via “set_bus”, which indirectly affects line connectivity based on topological configurations. To maintain consistent action semantics and simplify the learning process, we customized the agent’s action space to use “Set” actions (“set_line_status” and “set_bus”), providing deterministic state transitions that yield more stable policy learning compared to the toggle-based “Change” operations. These actions encompass a wide range of grid management capabilities, including line status modification, topological alterations, power redispatch, load curtailment, and management of storage units. This framework also supports predefined and custom reward functions for performance evaluation, such as CloseToOverflowReward, which penalizes states where lines approach overflow conditions, and LinesCapacityReward, which evaluates the optimal use of line capacity.

Some environments include opponents that simulate disruptive attacks. In our simulated environment, the adversary quasi-randomly performs line-tripping attacks. Specifically, the adversary employs a weighted random selection mechanism where line-tripping probabilities are proportional to their strategic importance ( $\rho$-normalized), creating realistic adversarial behavior that targets vulnerable infrastructure while maintaining stochasticity. The attacks occur with a 24-h cooldown period and a 4-h duration, subject to a budget constraint limiting their frequency and intensity.

For a direct comparison with trained solutions, Grid2Op supplies baseline agents. For instance, the “RandomAgent” takes random actions, while the “ExpertAgent” uses a greedy algorithm to streamline the overflow resolution process.

One critical challenge these agents must solve is preventing power grid capacity overload. It can occur due to a combination of factors, such as scheduled maintenance events, equipment faults, and malicious actions. Additionally, fluctuations in power demand, extreme weather conditions, and inadequate infrastructure can also exacerbate such incidents. It is indispensable that when we train control agents, they must be capable of proficiently managing overflow situations using the actions available. Figure 3 illustrates a bus-splitting action in response to an overflow scenario within a simulated Grid2Op environment. As depicted on the left side of the Fig. 3, at timestep $t$, the grid experiences line congestion of 118 MW on the eastern transmission path (shown in red), exceeding operational limits due to aggregate power demands of −150 and −100 MV at the load sinks. The implemented topological action involves bus-splitting at the receiving substation, creating parallel power flow paths. This reconfiguration at $t+1$, as visualized on the right side of Fig. 3, redistributes power flows more optimally across the network, reducing maximum line loading to 100 MW while maintaining constant generation dispatch (50, 200 MV) and load consumption patterns. The bus-splitting intervention successfully mitigated the congestion without requiring changes to generation schedules or load curtailment, which can be costly and detrimental to system reliability.

CPLF’s reward function

We define the composite reward function $R_t$ at timestep $t$ as a weighted linear combination of four reward components:

(1) $$R_t=w_1\cdot R_t^{N-1}+w_2\cdot R_t^{game}+w_3\cdot R_t^{capacity}+w_4\cdot R_t^{alert}$$where the weight vector $\mathbf{w}=[3.0,2.0,1.5,1.0]$ establishes a hierarchical priority structure that emphasizes security criteria over operational efficiency.

Implementation and evaluation protocol

We rigorously assess the proposed CPLF framework using a scenario-based evaluation protocol within the Grid2Op simulation environment. Our primary evaluation metric is the success rate. Table 4 formally defines all key performance indicators (KPIs) used throughout our evaluation. This metric, as specified in Eq. (6), quantifies resilience as the percentage of simulated scenarios in which agents maintain grid stability for a minimum duration of 288 timesteps, equivalent to 24 h of continuous operation. This duration captures a complete daily load cycle, creating a meaningful timeframe to assess the agent’s performance across varying operational conditions, including peak and off-peak demand periods. The 24-h evaluation window allows for observation of agent behavior under typical fluctuations in electricity demand and traditional generation that occur throughout the day, establishing it as a practical benchmark for testing grid management strategies in simulation environments.

(6) $$SuccessRate=\frac{\mathrm{N}\mathrm{u}\mathrm{m}\mathrm{b}\mathrm{e}\mathrm{r}\mathrm{o}\mathrm{f}\mathrm{s}\mathrm{c}\mathrm{e}\mathrm{n}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{o}\mathrm{s}\mathrm{s}\mathrm{u}\mathrm{r}\mathrm{v}\mathrm{i}\mathrm{v}\mathrm{i}\mathrm{n}\mathrm{g}\ge 288\mathrm{t}\mathrm{i}\mathrm{m}\mathrm{e}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{p}\mathrm{s}}{\mathrm{T}\mathrm{o}\mathrm{t}\mathrm{a}\mathrm{l}\mathrm{s}\mathrm{c}\mathrm{e}\mathrm{n}\mathrm{a}\mathrm{r}\mathrm{i}\mathrm{o}\mathrm{s}}\times 100\mathrm{\%}.$$

Grid stability maintenance requires preventing both thermal overloads ( $\rho >1.0$ for extended periods) and topological instabilities that result in disconnected loads or unsustainable power flow distributions. The evaluation encompasses 100 distinct adversarial test scenarios, each executing for a maximum of 2,000 timesteps with systematically varied environmental and agent seed values to achieve methodological robustness and reproducible results.

We implemented systematic seed management across different testing phases. The environment evaluation utilized seeds 300–399, while algorithm-specific evaluations employed distinct seed ranges: the PPO phase (150–249), the TRPO phase (200–299), the A2C phase (300–399), and reference implementations (400–499). The observation space, detailed in Table 3, included power flow metrics (P, Q, V, A) for transmission line monitoring, generator output and load demand states, network topological configurations, line capacity utilization coefficients ( $\rho$), and temporal indicators for maintenance scheduling and cooldown periods.

Through systematic hyperparameter optimization, we developed a neural network architecture featuring two fully connected hidden layers with 1,200 and 1,000 units, respectively, utilizing LeakyReLU activation functions ( $\alpha =0.01$). The network implemented a custom Actor-Critic Policy. We leveraged Stable Baselines 3 for implementing PPO, A2C, and TRPO algorithms (Raffin et al., 2021). All experiments were conducted on an NVIDIA Tesla P100 GPU platform. Table 5 outlines the complete set of hyperparameters used across all algorithms in our framework (PPO, TRPO, A2C), including network architecture specifications and algorithm-specific parameters that we tuned to attain good performance and stable policy transfer.

Our training protocol encompassed several configurations to assess the performance of different algorithmic combinations:

• Single algorithm training: All configurations utilized GAE- $\lambda$ ( $\lambda =0.95$) for advantage estimation:

  • –

    PPO: Trained for 16,000 iterations with a learning rate of $3\times {10}^{-4}$.

  • –

    A2C: Trained for 16,000 iterations using a learning rate of $7\times {10}^{-4}$.

  • –

    TRPO: Trained for 16,000 iterations using a learning rate of $7\times {10}^{-4}$.

• Single pre-trained baselines:

  • – Baseline 1 (A2C $\to$ PPO): A2C was initially trained for 9,600 iterations, followed by PPO training for 9,600, transferring the learned policy from A2C to PPO.

  • – Baseline 2 (PPO $\to$ A2C): PPO trained for 9,600 iterations. Subsequently, A2C leverages this initial policy and further fine-tunes it for 6,000, culminating in a more stabilized policy.

  • – Baseline 3 (PPO $\to$ TRPO): After an initial 9,600 iterations with PPO, TRPO continued training for 6,000 iterations, incorporating the preceding policy to achieve greater stability.

• Double-stage cascading baselines: These configurations investigate the cumulative effect of cascading policy knowledge through three distinct DRL algorithms to establish ideal transfer sequences. All baselines maintain consistent neural network architectures during knowledge transfer to sustain stable policy inheritance.

  • – Baseline 4 (TRPO $\to$ PPO $\to$ A2C): TRPO trains for a specified number of iterations, then transfers its learned policy to PPO for further training. Finally, A2C fine-tunes the resulting policy, building upon the knowledge from both preceding stages.

  • – Baseline 5 (A2C $\to$ PPO $\to$ TRPO): A2C initiates training and transfers its policy to PPO for intermediate optimization. PPO then transfers the refined policy to TRPO for final optimization.

  • – Baseline 6 (PPO $\to$ A2C $\to$ TRPO): PPO establishes an initial policy and transfers it to A2C for intermediate tuning. TRPO then performs the final optimization stage using the policy parameters from A2C.

  • – Baseline 7 (A2C $\to$ TRPO $\to$ PPO): A2C establishes an initial policy and transfers it to TRPO for intermediate refinement with monotonic improvement guarantees. PPO then performs the final optimization stage using the policy inherited from TRPO.

  • – Baseline 8 (TRPO $\to$ A2C $\to$ PPO): TRPO initiates training to ensure stable policy development, then transfers the policy to A2C for efficient intermediate training. A2C transfers the refined policy to PPO for final fine-tuning and deployment.

• Framework training: Our cascading framework implements a three-stage training process. Each stage inherits and builds upon the policy learned in previous stages, as in Fig. 4:

  • – Stage 1: PPO training for 9,600 iterations to establish robust baseline policies.

  • – Stage 2: TRPO optimization for 1,600 iterations, delivering conservative policy updates while maintaining performance.

  • – Stage 3: A2C fine-tuning for 3,200 iterations, focusing on sample-efficient policy improvement.

The computational requirements for CPLF implementation vary vastly across training stages. PPO training required approximately one hour on an NVIDIA Tesla P100 GPU platform for 11,200 iterations, while TRPO’s conservative update mechanism completed 1,600 iterations in 6 h. A2C fine-tuning proved the most resource-light, requiring only 1 h for 3,200 iterations due to its streamlined architecture and reduced batch size requirements.

Memory usage peaked at 16 GB during TRPO training phases, while PPO and A2C maintained more modest memory footprints of 8 and 4 GB, respectively. The training data pipeline benefits from Grid2Op’s structured format, requiring minimal preprocessing overhead. State normalization occurs automatically within the environment, and the standardized observation space removes the requirement for custom feature engineering or data transformation procedures that introduce computational bottlenecks or systematic biases.

Performance analysis

Our empirical evaluation revealed several pronounced findings regarding the framework’s performance. The survival analysis, depicted in Fig. 5, shows substantial variance across scenarios, with numerous instances exceeding 1,500 timesteps of stable operation. The visualization plots survival duration (0–2,000 timesteps) against scenario indices (0–100), with a threshold line (presented by the red line) at 288 timesteps, establishing our minimum success criterion.

To illustrate the contribution of each algorithmic stage to the CPLF’s performance, we analyzed the framework’s learning progression over the entire training timeline. Figure 6 illustrates this process, plotting the average number of survival steps achieved as the framework transitions through its three sequential stages.

The cascading process unfolds as follows:

1. Stage 1: PPO Foundation (0–11,200 timesteps) The framework begins with Proximal Policy Optimization (PPO) to establish a stable policy foundation. As shown in Fig. 6, PPO steadily improves performance from a low baseline, creating a robust but modest initial policy that reaches an average survival of 107.9 steps. This initial phase is a prerequisite for exploring the high-dimensional action space without catastrophic policy collapses.

2. Stage 2: TRPO (13,000–14,500 timesteps) Upon inheriting the policy from PPO, the Trust Region Policy Optimization (TRPO) stage commences. This transition yielded an immediate and substantial performance boost. It validates our choice of TRPO, given its mathematically rigorous updates that optimize the existing policy within a constrained trust region, thereby guaranteeing monotonic improvement. This stage elevates the policy’s performance to an average of 191.1 survival steps.

3. Stage 3: A2C fine-tuning (14,500–17,500 timesteps) In the final stage, we transfer the polished policy to the Advantage Actor-Critic agent for policy tuning. A2C’s ability to use a learned value function baseline empowers more precise and sample-efficient policy improvements. The agent’s performance increases in this final phase, as it decisively and consistently exceeds the 288-timestep survival target required for success, ultimately achieving an average of 394.1 survival steps.

This sequential, stage-wise visualization confirms that the CPLF’s superior performance is not incidental but a direct result of its cascading design. As illustrated in the detailed workflow presented in Fig. 4, each algorithm builds upon the distilled knowledge of its predecessor, validating our hypothesis that sequential knowledge transfer across complementary algorithms yields a more robust control policy for power grid resilience. The subsequent analysis compares the final performance of this complete framework against a range of baseline configurations across 100 adversarial scenarios. Due to computational constraints ( $\approx$8 h per training run on NVIDIA P100 GPUs), we trained each permutation once; however, we evaluated each resulting policy across 100 diverse random adversarial scenarios (seeds 300–399) to ensure statistical robustness of our performance comparisons.

To quantify the individual contributions of each algorithmic component, we conducted a complete ablation study examining the compelling performance hierarchy across different baselines. As depicted in Fig. 7, each data point represents the mean survival duration (0–2,000 timesteps) over 100 scenarios. Our rationale for the PPO $\to$ TRPO $\to$ A2C sequence is grounded in algorithmic complementarity: PPO’s clipped surrogate objective fosters stable initial policy development, which is a prerequisite for success in high-dimensional power grid environments, TRPO’s KL divergence constraint guarantees mathematically rigorous updates by preserving well-established policy patterns, and A2C’s value function baseline achieves precise policy improvements through reduced gradient variance. The results reveal clear evidence for the necessity of each element in achieving optimal performance.

Single-algorithm baselines revealed severe limitations in managing adversarial power grid scenarios. The Random Agent baseline and standalone PPO both achieved 0% success rates, failing to maintain grid stability beyond the minimum threshold in any test scenario. PPO’s failure stems from its exploration strategy, which proves inadequate for the highly constrained and safety-critical power grid environment. A standalone TRPO achieved only a 2% success rate (average survival: 34.6 steps, median: 27.0 steps), demonstrating that conservative policy updates alone are insufficient without a stable initial foundation. Standalone A2C performed moderately better with a 34% success rate, benefiting from its value function baseline but still lacking the robustness required for consistent performance.

The introduction of policy transfer between algorithms resulted in meaningful improvements, validating the utility of knowledge transfer between complementary algorithms. The PPO $\to$ TRPO configuration achieved a 39% success rate, with an average survival of 462.7 steps and a median of 193.0 steps, suggesting that TRPO’s conservative optimization can build upon PPO’s foundation but remains limited without subsequent fine-tuning. The PPO $\to$ A2C configuration reached a 43% success rate (average survival: 469.2 steps, median: 205.5 steps), confirming the benefit of PPO’s initial policy establishment, followed by A2C’s optimized fine-tuning capabilities.

Alternative three-stage configurations further illustrated the imperative of proper algorithmic sequencing. The TRPO $\to$ PPO $\to$ A2C sequence achieved a 52% success rate, with an average survival of 418.5 steps and a median of 319.0 steps. This reduction stemmed from the TRPO’s conservative nature and proved inadequate for initial policy exploration in the high-dimensional Grid2Op environment. A2C $\to$ PPO $\to$ TRPO reached a 33% success rate (average survival: 350.8 steps, median: 98.5 steps), because the unstable foundations created by A2C’s high-variance initial learning hindered subsequent algorithms from achieving stability. PPO $\to$ A2C $\to$ TRPO achieved a 42% success rate (average survival: 404.3 steps, median: 111.5 steps), with TRPO’s conservative adjustments in the final position unable to fully capitalize on the A2C intermediate stage. Table 6 provides a detailed statistical analysis, including the mean, standard deviation, median, 95th percentile, coefficient of variation, and performance stability metrics for all multi-stage configurations (statistics achieved by our solution, CPLF, are shown in bold). These results confirm that initiating the cascade with PPO’s stable foundation is a prerequisite and that TRPO’s conservative optimization works best as an intermediate stage rather than as a final component.

The complete CPLF framework, which extends the knowledge transfer concept by incorporating TRPO as an intermediate optimization stage, achieved an exceptional 84% success rate with an average survival of 861.0 steps and a median of 786.0 steps. As shown in Fig. 8, the framework’s robust performance is further evidenced by its superior survival duration distribution, with the highest median survival and most consistent performance across all scenarios. The framework’s 95th percentile survival duration of 1,957.2 steps reflects consistent high performance across challenging scenarios, distinguishing it from the more variable performance patterns exhibited by other configurations. This framework leverages TRPO’s ability to deliver mathematically rigorous updates, building upon the initial policy established by PPO and establishing a strong foundation for final tuning by A2C. The 41-percentage-point improvement over the best two-stage configuration (PPO $\to$ A2C: 43%) reveals that the additional architectural complexity yields meaningful advantages in grid stability management. The progressive performance improvements from single-algorithm (0–34%) to two-stage transfers (39–43%) and finally to the complete CPLF framework (84%) substantiate the core premise of our cascading approach: that sequential knowledge transfer across multiple algorithmic stages delivers markedly superior policy development compared to simpler architectural configurations.

Figure 9 illustrates the relationship between success rate and survival duration across all configurations, with the complete quantitative results detailed in Table 7. Our CPLF framework uniquely occupies the optimal performance quadrant, achieving both a high 84% success rate and an extended 861-timestep survival duration, thereby establishing a new benchmark for power grid resilience under adversarial conditions. While most other configurations exhibit clear trade-offs between these metrics, the progressive performance improvements from single-algorithm (0–34%) to two-stage transfers (39–43%) and finally to the complete CPLF framework (84%) substantiate the pivotal premise of our cascading approach: sequential knowledge transfer across multiple algorithmic stages delivers markedly superior policy development compared to simpler architectural configurations.

Our experimental design mitigates the risk of overfitting by training on “level 2” scenarios and evaluating 100 unseen, more challenging “competition level” scenarios. The framework’s 84% success rate on this holdout set affirms strong generalization. Training stability stemmed from the inherent properties of the cascaded algorithms (PPO’s clipping, TRPO’s KL-divergence constraint, and A2C’s variance reduction). The monotonic performance improvement and smooth convergence shown in Fig. 6 corroborate the successful implementation of this approach.

While the 84% success rate demonstrates strong overall performance, analyzing the 16 failures provides valuable insights into the framework’s operational boundaries and remaining vulnerabilities. To complement the success rate metric and provide deeper insights into resilience characteristics, we conducted a failure analysis examining constraint violations across all unsuccessful episodes. This analysis reveals two primary failure patterns that account for all 16 constraint violations (16 failed scenarios out of 100 total). The dominant failure mode, occurring in 10 of these cases, involved concurrent high-impact contingencies where the adversary strategically initiated an attack on a critical transmission line immediately followed by a scheduled maintenance outage on a nearby parallel line. This N − 2 situation created a severe topological bottleneck that the agent could not resolve sufficiently quickly, ultimately leading to cascading overloads. The remaining six failures exhibited a distinct pattern characterized by substation isolation, where coordinated attacks on multiple lines connected to a single substation effectively severed it from the central grid, resulting in immediate blackouts for the loads it served.