AlphaViT: a flexible game-playing AI for multiple games and variable board sizes

Architectures of DNNs

AlphaViT An overview of AlphaViT’s DNN architecture is shown in Fig. 2. The DNN in AlphaViT is based on ViT, which has no input-size limitation and can classify images even if the input image size differs from the training image size. This flexibility enables AlphaViT to play games with different board sizes using the same network. In AlphaViT, the game boards are fed into ViT. Initially, these inputs are transformed into patch embeddings through a convolutional layer. Using a convolutional layer allows easy adjustment of the patch partitioning parameters, such as patch size, stride, and padding. The final output of the encoder is used to compute the value and move probabilities. AlphaViT employs three special tokens to enable flexibility for different games and board sizes:

• Value token ${\mathbf{x}}_{\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}}$: is fed to an MLP value head to predict the state value $v(s)\in (-1,1)$, where $s$ is the game state, $1$ indicates a certain win, and $-1$ a certain loss. This token is learnable.

• Game token ${\mathbf{x}}_{\mathrm{g}\mathrm{a}\mathrm{m}\mathrm{e}}$: encodes which game is currently being played. It is represented as a non-trainable one-hot vector whose length is equal to the embedding size D. Each game type (e.g., Connect 4, Gomoku, Othello) is assigned a unique index in this vector. Although the token itself is fixed, the model can learn game-specific representations through downstream layers. This design also enables flexible addition of new games, up to D in number.

• Pass token ${\mathbf{x}}_{\mathrm{p}\mathrm{a}\mathrm{s}\mathrm{s}}$: represents the “pass” action that exists in Othello but not in Gomoku or Connect 4. Appending a dedicated learnable token vector after all patch embeddings, this enables the model to predict the probability of the pass move when the rules require it. This token is learnable.

Given a game state $s$, AlphaViT’s DNN predicts the value $v(s)$ and the move probability vector $\mathit{p}(s)$, which includes $p(a\mid s)$ for each action $a$. In a board game, $s$ represents the board state, and $a$ denotes a move.

The input to the DNN is an $H\times W\times (2T+1)$ image stack $\mathbf{x}\in {\mathbb{R}}^{H\times W\times (2T+1)}$, consisting of $2T+1$ binary feature planes of size $H\times W$. Here, H and W are the dimensions of the board, and T is the number of history planes. The first T feature planes represent the occupancies of the first player’s discs, where a feature value of 1 means that a disc occupies the corresponding cell, and 0 means it does not. The following T feature planes represent the occupancies of the second player’s discs. The final feature plane represents the color of the current player’s disc, where 1 denotes the first player and −1 denotes the second player.

The convolutional layer divides the image stack $\mathbf{x}$ into $P\times P$ patches with stride $\mathrm{s}\mathrm{t}\mathrm{r}$ and padding $\mathrm{p}\mathrm{a}\mathrm{d}$, and generates the patch embeddings ${\mathbf{z}}_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}$. The patch size P corresponds to the kernel size of the convolutional layer. The sequence of patch embeddings ${\mathbf{z}}_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}$ is defined in Eq. (1) as follows:

(1) $${\mathbf{z}}_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}=[{\mathbf{x}}_p^0\mathbf{E};\dots ;{\mathbf{x}}_p^i\mathbf{E};\dots ;{\mathbf{x}}_p^{N_p-1}\mathbf{E}],$$where $N_p$ is the number of patches, ${\mathbf{x}}_p^i\in {\mathbb{R}}^{P^2(2T+1)}$ is the $i$th flattened 2D patch, and $\mathbf{E}$ is a trainable embedding tensor with the shape $(P^2(2T+1),D)$. Here, $N_p=\left(\lfloor H+2\text{\hspace{0.17em}pad}-P\text{str}\rfloor +1\right)\times \left(\lfloor W+2\text{\hspace{0.17em}pad}-P\text{str}\rfloor +1\right)$, where H and W are the board height and width, P is the patch size, $\mathrm{s}\mathrm{t}\mathrm{r}$ is the stride, and $\mathrm{p}\mathrm{a}\mathrm{d}$ is the padding used in the convolutional layer. The kernel of the convolutional layer acts as the tensor $\mathbf{E}$, which maps each patch to a D-dimensional embedding space. Here, D is the embedding size.

To retain positional information, learnable 2D positional embeddings ${\mathbf{E}}_{\mathrm{p}\mathrm{o}\mathrm{s}}$ are added to the patch embeddings. These positional embeddings are scaled according to the board size and hyperparameters to match the size of the patch embeddings ${\mathbf{z}}_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}$. The resulting position-aware patch embeddings are given by Eq. (2):

(2) $${\mathbf{z}}_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}^{\mathrm{p}\mathrm{o}\mathrm{s}}={\mathbf{z}}_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}+{\mathbf{E}}_{\mathrm{p}\mathrm{o}\mathrm{s}}.$$

The output size of the transformer encoder is determined by the number of input tokens. For Gomoku, where the action space is HW, AlphaViT requires HW embeddings (i.e., $N_p=HW$). To achieve this, the patch size P is set to $2k+1$, where $k$ is a non-negative integer, stride $\mathrm{s}\mathrm{t}\mathrm{r}$ is set to $1$, and padding $\mathrm{p}\mathrm{a}\mathrm{d}$ is set to $\lfloor P/2\rfloor$. Note that when $k>0$, patches overlap, but the total number of patches remains HW. In contrast, for Othello, the action space is $HW+1$ to include the pass move. Using the same parameters as Gomoku ( $P=2k+1$, $\mathrm{s}\mathrm{t}\mathrm{r}=1$, and $\mathrm{p}\mathrm{a}\mathrm{d}=\lfloor P/2\rfloor$), AlphaViT requires one additional token for the pass move. To address this, a learnable pass token ${\mathbf{x}}_{\mathrm{p}\mathrm{a}\mathrm{s}\mathrm{s}}$ is introduced by appending it after all patch embeddings (see Eq. (3)). To estimate the board value, a learnable value token ${\mathbf{x}}_{\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}}$ is prepended. Additionally, to enable AlphaViT to recognize different game types, a non-trainable one-hot game token ${\mathbf{x}}_{\mathrm{g}\mathrm{a}\mathrm{m}\mathrm{e}}$, represented using one-hot encoding, is incorporated. These tokens are appended to the embeddings ${\mathbf{z}}_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}^{\mathrm{p}\mathrm{o}\mathrm{s}}$. As a result, the input embeddings ${\mathbf{z}}_0$ to the transformer encoder are defined in Eq. (3) as follows:

(3) $${\mathbf{z}}_0=[{\mathbf{x}}_{\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}};{\mathbf{x}}_{\mathrm{g}\mathrm{a}\mathrm{m}\mathrm{e}};{\mathbf{z}}_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}^{\mathrm{p}\mathrm{o}\mathrm{s}};{\mathbf{x}}_{\mathrm{p}\mathrm{a}\mathrm{s}\mathrm{s}}].$$

Positional embeddings are added to the patch embeddings before appending the pass, value, and game embeddings. This approach ensures that the positional embeddings can be scaled independently, without being affected by embeddings that do not inherently contain positional information.

The sequence ${\mathbf{z}}_0$ is fed into the transformer encoder, which consists of L transformer encoder layers. The output of the final encoder layer ${\mathbf{z}}_L$ has shape $(HW+3)\times D$. The first vector ${\mathbf{z}}_L^0$ derived from the value token is processed by the value head implemented as an MLP denoted as $\mathrm{M}\mathrm{L}{\mathrm{P}}_v$. This head estimates the value $v$ using Eq. (4):

(4) $$v=\tanh (\mathrm{M}\mathrm{L}{\mathrm{P}}_v(\mathrm{L}\mathrm{N}({\mathbf{z}}_L^0))),$$where LN represents layer normalization. The tanh activation constrains the value within the range $(-1,1)$, representing the state value, where $1$ indicates a certain win and $-1$ a certain loss.

The vectors ${\mathbf{z}}_p=[{\mathbf{z}}_L^2,\dots ,{\mathbf{z}}_L^{HW+2}]$ (index 0: value, 1: game, $2\dots HW+1$: patches, $HW+2$: pass), derived from the board patches and the pass token, are processed by the policy head, which is implemented as another MLP ( $\mathrm{M}\mathrm{L}{\mathrm{P}}_p$). The $H\times W$ board positions and the pass move are flattened into one-dimensional indices $i\in \{0,1,\dots ,HW\}$. For $i<HW$, the move corresponds to board coordinates $(m,n)$ where $m=imodW$ is the column index, and $n=\lfloor i/W\rfloor$ is the row index. The policy head applies a shared MLP to each token, and a softmax over the resulting logits yields a probability vector $\mathit{p}(s)\in {\mathbb{R}}^{HW+1}$ whose $i$th entry is $p(a_i\mid s)$. The special index $i=HW$ represents the pass action. Thus, the policy head output is defined in Eq. (5):

(5) $$p(a_i\mid s)=\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}(\mathrm{M}\mathrm{L}{\mathrm{P}}_p(\mathrm{L}\mathrm{N}({\mathbf{z}}_p)){)}_i,$$where $a_i$ is the $i$ th action, and the MLP is applied to each of the $HW+1$ tokens. For Othello, the probability for the pass move is $p(a_{HW}\mid s)$. In Gomoku, this entry is masked out since the pass move is not allowed. In Connect 4, only the probabilities $\{p(a_i\mid s)\mid 0\le i<W\}$ (i.e., moves $(m=i,n=0)$) corresponding to valid column choices $i$ are used and renormalized. Note that $i$ is the column index in Connect 4. The agent drops the disc into column $i$, and the disc then falls to the lowest available row.

To decide on a move, AlphaViT employs MCTS with Upper Confidence Bound applied to Trees (UCT), using the value and move probabilities computed by the DNN. The MCTS algorithm used in AlphaViT is identical to that of AlphaZero, as detailed in ‘AlphaZero’. The hyperparameters for AlphaViT are listed in ‘Parameters’.

AlphaViD and AlphaVDA AlphaViT’s DNN has a significant drawback: the size of the move probability vector (policy) is fixed by the transformer encoder’s input size. To address this issue, AlphaViD and AlphaVDA incorporate both a transformer encoder and a decoder for calculating value and move probabilities, as shown in Fig. 3. In AlphaViD, the decoder receives input derived from the output of the encoder. In contrast, AlphaVDA employs learnable action embeddings as inputs to the decoder. The value is computed from the encoder’s final output, while the move probabilities are computed from the final output of the decoder.

In AlphaViD and AlphaVDA, the input board is linearly embedded using a convolutional layer and fed into a transformer encoder, similar to AlphaViT. However, unlike AlphaViT, no pass token is included in the encoder input. The input embedding sequence is defined in Eq. (6) as follows:

(6) $${\mathbf{z}}_0=[{\mathbf{x}}_{\mathrm{v}\mathrm{a}\mathrm{l}\mathrm{u}\mathrm{e}};{\mathbf{x}}_{\mathrm{g}\mathrm{a}\mathrm{m}\mathrm{e}};{\mathbf{z}}_{\mathrm{p}\mathrm{a}\mathrm{t}\mathrm{c}\mathrm{h}}^{\mathrm{p}\mathrm{o}\mathrm{s}}],$$where the sequence consists of a value token, a game token, and patch embeddings. The estimated value is obtained from the value head that processes the output embedding corresponding to the value token from the last layer of the transformer encoder. In both AlphaViD and AlphaVDA, the encoder output is only used directly for value estimation, indicating that the number of patch embeddings $N_p$ need not match the action-space size.

The architecture of AlphaViD’s DNN is shown at the left of Fig. 3. The DNN estimates the move probability vector using the transformer decoder and MLP_p. The input embeddings for the transformer decoder are derived from the outputs of the transformer encoder corresponding to the patch embeddings, which are further processed through a fully connected layer. The initial embeddings for the decoder input are defined in Eq. (7) as follows:

(7) $${\mathbf{y}}_0^{\mathrm{\prime }}=\mathrm{M}\mathrm{L}\mathrm{P}([{\mathbf{z}}_L^2;\dots ;{\mathbf{z}}_L^{N_p+1}]),{{\mathbf{y}}^{\mathrm{\prime }}}_0\in {\mathbb{R}}^{N_p\times D_d},$$where $D_d$ is the embedding size of the decoder. Since the embedding sequence size must match the input size of the transformer decoder, ${\mathbf{y}}_0^{\mathrm{\prime }}$ is interpolated to ${\mathbf{y}}_0\in {\mathbb{R}}^{N_a\times D_d}$, where $N_a$ is the action space size. In this study, this step uses bilinear interpolation in the function torch.nn.functional.interpolate. This interpolation provides flexibility to adjust the action space size depending on the game type and board size. If an additional move such as a “pass” (e.g., in Othello) is required, $N_a$ is set to $HW+1$ and ${\mathbf{y}}_0^{\mathrm{\prime }}$ is resized accordingly via this interpolation. The additional embedding at the last position is assigned to the additional move (e.g., the pass). Therefore, a dedicated pass token is not required for the encoder’s input. Similar to the original transformer, the transformer decoder receives ${\mathbf{y}}_0$ as the target sequence and ${\mathbf{z}}_L$ as the memory (encoder output). Finally, MLP_p calculates the move probabilities from the output of the last layer of the transformer decoder ${\mathbf{y}}_L$. The probability of action $a_i$ is given by Eq. (8):

(8) $$p(a_i\mid s)=\mathrm{S}\mathrm{o}\mathrm{f}\mathrm{t}\mathrm{m}\mathrm{a}\mathrm{x}(\mathrm{M}\mathrm{L}{\mathrm{P}}_p(\mathrm{L}\mathrm{N}({\mathbf{y}}_L)){)}_i.$$

The architecture of AlphaVDA’s DNN is shown at the right of Fig. 3. The DNN is similar to that of AlphaViD, but uses learnable embeddings ${\mathbf{y}}_0^{\mathrm{\prime }}$ as the initial embeddings for the transformer decoder. While the length of these learnable embeddings is fixed, they are interpolated to the decoder input ${\mathbf{y}}_0$ to match the action space size, ensuring compatibility and adaptability across different game configurations.

To decide on a move, AlphaViD and AlphaVDA employ MCTS with UCT, using the value and move probabilities predicted by their DNNs. The MCTS algorithm implemented in these agents is identical to that of AlphaZero, as detailed in ‘AlphaZero’. The hyperparameters for AlphaViD and AlphaVDA are listed in ‘Parameters’.

Games

This study evaluates the proposed agents on six game variants (three different games, each played on two board sizes): Connect 4 ( $7\times 6$; “Connect 4”, and $5\times 4$; “Connect 4 $5\times 4$”), Gomoku ( $9\times 9$; “Gomoku”, and $6\times 6$; “Gomoku $6\times 6$”), and Othello ( $8\times 8$; “Othello”, and $6\times 6$; “Othello $6\times 6$”). These games are two-player, deterministic, zero-sum games with perfect information. Connect 4 is a connection game played on a $7\times 6$ board, published by Milton Bradley. The players take turns dropping discs onto the board. A player wins by forming a straight line of four discs horizontally, vertically, or diagonally. Connect 4 $5\times 4$ is a variant of Connect 4 on a $5\times 4$ board. Gomoku is a connection game in which players place stones on a board to form a straight line of five stones, either horizontally, vertically, or diagonally. This study uses a $9\times 9$ board for Gomoku and a $6\times 6$ board for Gomoku $6\times 6$. Othello (also known as Reversi) is a two-player strategy game played on an $8\times 8$ board. In Othello, the discs are white on one side and black on the other. Players take turns placing a disc with their assigned color facing up. During a game, discs of the opponent’s color are flipped to the current player’s color if they are in a straight line and bounded by the disc just placed and another disc of the current player’s color. Othello 6 × 6 is played on a $6\times 6$ board in this study. These games were selected (i) each has natural board-size variants (5 × 4 – 9 × 9), (ii) they are commonly used for benchmarking, and (iii) training can be completed on commodity GPUs. The description of the game rules is partly adapted from our previous publication (Fujita, 2022).

Opponents

This study evaluates the performance of AlphaViT, AlphaViD, and AlphaVDA using five different AI methods: AlphaZero, two variants of MCTS labeled MCTS100 and MCTS400, Minimax, and Random. AlphaZero was trained using the method described in ‘AlphaZero’. The MCTS methods (MCTS100 and MCTS400) were run with 100 and 400 simulations, respectively.

The details of MCTS are provided in the Supplemental Material (Sec. S1). In these MCTS methods, the child nodes are expanded on the fifth visit to a node. Minimax selects a move using the minimax algorithm based on the evaluation table described in the Supplemental Material (Sec. S2). The Random agent selects moves uniformly at random from the set of valid moves.

Previous studies indicate that vanilla MCTS with random rollouts scales poorly, remaining weaker than a shallow $\alpha$– $\beta$ search even with an increased number of simulations. For instance, in Connect-4, a UCT agent employing $\mathrm{10,000}$ random simulations achieved only a $19.8\mathrm{\%}$ win rate against a depth-8 $\alpha$– $\beta$ opponent (Scheiermann & Konen, 2023). In contrast, incorporating domain-specific heuristics such as “decisive-move” pruning substantially improves MCTS performance, often yielding strength increases of one to two orders of magnitude (Teytaud & Teytaud, 2010; Taylor & Stella, 2024). The measurements in this study (the Supplemental Material, Sec. S3) corroborate these findings for MCTS, showing rapid Elo improvement up to approximately $400$ simulations followed by diminishing returns. Therefore, this study adopts $100$ and $400$ simulations as the MCTS baselines, providing clearly separated strength levels while maintaining reasonable computational requirements.

Software

AlphaViT, AlphaViD, AlphaVDA, the opponents, and the board games were implemented in Python, using NumPy for linear algebra operations and PyTorch for DNNs. The source code and the raw data are available on GitHub at https://github.com/KazuhisaFujita/AlphaViT for reproducibility and further extensions of this work.

Hardware

All experiments were conducted on multiple custom-built PCs. The agents were run on a variety of consumer CPUs, including AMD Ryzen 9 9900X and Intel Core i7-14700K, Core i9-13900K, and Core i9-12900K processors. The GPUs used for the experiments were NVIDIA GeForce RTX 4060 Ti (16 GB), RTX 3060 (12 GB), RTX 2070 Super, and RTX 2070. Each machine was equipped with 64 GB of RAM and two GPUs, and the experiments were run on Debian Linux. Model training and evaluation were distributed across multiple GPUs on a single machine using data parallelism. No cloud-based or high-performance computing clusters were used, and all computations were performed on local custom-built hardware. For example, one of the machines used for training and evaluation was equipped with an Intel Core i9-13900K CPU, 64 GB of RAM, and two NVIDIA GeForce RTX 4060 Ti (16 GB) GPUs.

Baseline Elo ratings

Objective and Setup. Tables 3, 4, and 5 present Elo ratings of various AI agents across different games and board sizes. Elo ratings provide a standard measure of relative performance in two-player games and enable systematic comparison across agents. This study evaluated multiple variants of the proposed models, including agents trained on a single game with either large or small boards, and multiple games with large boards. Comparisons were conducted with other AI agents, including AlphaZero, MCTS with different numbers of simulations, Minimax, and a Random agent. The proposed agents and AlphaZero underwent 1,000 training iterations; each iteration consisted of self-play, augmentation, and an update step detailed in ‘Training procedure’. The Elo ratings of all agents were initialized to 1,500 and were calculated through 50 round-robin tournaments between the agents. Each Elo rating is accompanied by a 95% confidence interval (95% CI); the calculation details are provided in the Supplemental Material (Sec. S4).

Note: Throughout this article, the term strong performance refers to benchmark-relative performance. An agent is considered strong if its Elo rating exceeds the baseline AlphaZero’s Elo rating minus 100 (as shown in Tables 3, 4, and 5). Thus, strong performance does not denote an absolute numerical threshold, but rather performance at the level of AlphaZero.

Highlights across games. Single-task AlphaViT L4 approaches AlphaZero’s performance across all evaluated games, remaining within approximately 270 Elo points of AlphaZero. Single-task AlphaViD L1 and AlphaVDA L1 achieve AlphaZero-level performance only on smaller boards (Connect 4 5 × 4 and Gomoku 6 × 6), yet they experience substantial drops in Elo ratings on larger boards. Multitask AlphaViT L4 (AlphaViT L4 Multi) demonstrates competitive performance on large boards and closely approaches AlphaZero (within 200 Elo points). Conversely, multitask AlphaViD L1 and AlphaVDA L1 variants exhibit significantly lower performance. Note that the multitask models were not trained on small board configurations and therefore underperform on these board sizes.

Connect 4 (Table 3). For standard Connect 4, AlphaViT L4 Multi most closely approaches AlphaZero ( $\mathrm{\Delta }$ Elo $\approx -160$). AlphaViT L4 LB is moderately weaker ( $\mathrm{\Delta }$ Elo $\approx -260$), whereas AlphaViD L1 LB and AlphaVDA L1 LB lag behind significantly ( $\mathrm{\Delta }$ Elo $<-300$). On the smaller 5 × 4 board, all single-task variants (AlphaViT L4 SB, AlphaViD L1 SB, AlphaVDA L1 SB) are competitive with AlphaZero.

Gomoku (Table 4). For the large board, only AlphaViTs (L4 LB and Multi) attain performance comparable to AlphaZero, with overlapping confidence intervals. AlphaViD and AlphaVDA variants (L1 LB and Multi) substantially lag behind ( $\mathrm{\Delta }$ Elo $<-300$). On the smaller board, SB variants from all architectures effectively approach AlphaZero-level performance.

Othello (Table 5). On the large board, AlphaViT L4 LB numerically surpasses AlphaZero ( $\mathrm{\Delta }$ Elo $\approx +20$), though overlapping confidence intervals prevent definitive conclusions. AlphaViD L1 LB and AlphaViT L4 Multi achieve performance near that of AlphaZero, with $\mathrm{\Delta }$ Elo $\approx -100$. AlphaVDA L1 LB lags behind significantly ( $\mathrm{\Delta }$ Elo $<-300$). On the smaller board, AlphaViT L4 SB closely approaches AlphaZero ( $\mathrm{\Delta }$ Elo $\approx -70$). AlphaViD L1 SB lags behind ( $\mathrm{\Delta }$ Elo $\approx -230$), whereas AlphaVDA L1 SB also falls short ( $\mathrm{\Delta }$ Elo $\approx -180$), remaining weaker than AlphaZero. Interestingly, AlphaViT L4 LB, despite being trained only on the large board, achieves performance comparable to AlphaViD L1 SB and AlphaVDA L1 SB on the smaller board. This suggests that knowledge may transfer from larger to smaller boards.

Conclusion

In summary, AlphaViT L4 and AlphaViT L4 Multi consistently achieve or closely approach AlphaZero-level strength across games, even without game-specific fine-tuning. In contrast, AlphaViD L1 and AlphaVDA L1 achieve AlphaZero-level performance only on smaller boards. The strong results for multitask agents (AlphaViT L4 Multi) suggest that multitask training will not significantly hinder performance. The Elo ratings presented in Table 3, 4, and 5 serve as a baseline for the subsequent experiments described in the following sections. Further analysis of AlphaZero’s architectural variants, including deeper networks, fine-tuning, and optimizer choices, is presented in the Supplemental Material (Sec. S5).

Variation of Elo rating over training iterations

Setup. Figure 4 illustrates the progression of Elo ratings for AlphaViT, AlphaViD, AlphaVDA, and AlphaZero across multiple training iterations for large and small board configurations in three games: Connect 4, Gomoku, and Othello. Elo ratings were recorded at every iteration from 1 to 10, subsequently every 20 iterations up to iteration 100, and thereafter at intervals of 100 iterations until iteration 3,000 for large board configurations and 1,000 for small board configurations. The different evaluation intervals were used because Elo ratings change more rapidly in the early training phase. The shaded areas around each Elo curve indicate the 95% confidence intervals computed through bootstrapping, as detailed in the Supplemental Material (Sec. S4). One iteration corresponds to a complete cycle of self-play data generation, data augmentation, and a single update of the neural network (see ‘Training procedure’). The opponents used for evaluation were identical to those described in previous experiments: AlphaViT L4 LB, AlphaViD L1 LB, AlphaVDA L1 LB, AlphaViT L4 SB, AlphaViD L1 SB, AlphaVDA L1 SB, AlphaViT L4 Multi, AlphaViD L1 Multi, AlphaVDA L1 Multi, AlphaZero, MCTS400, MCTS100, Minimax, and Random agents. The Elo ratings of these opponents were fixed as listed in Tables 3, 4, and 5. Evaluated agents’ Elo ratings were initialized to 1,500 and then calculated through 40-game matches against each opponent, consisting of 20 games as the first player and 20 games as the second player.

Large board configurations. Across all tested large board games, AlphaViT, AlphaViD, and AlphaVDA variants demonstrate rapid Elo improvement during the initial training phase, achieving approximately 80–90% of their maximum Elo within the first 300–500 iterations. In Gomoku, the single-task agents peak near 1,000 iterations and then exhibit a moderate decline. In Othello, the growth rate slows markedly after roughly 300 iterations, although the Elo ratings continue to increase slightly thereafter. Multitask-trained agents (AlphaViT Multi, AlphaViD Multi, AlphaVDA Multi) show learning speeds comparable to those of single-task models, confirming that multitask training does not adversely affect initial convergence. Notably, single-game-trained Gomoku models (AlphaViT L4 LB, AlphaViD L1 LB, AlphaVDA L1 LB) exhibit moderate Elo declines after reaching peak performance, indicating potential overfitting in later training stages. Conversely, multitask-trained Gomoku models (AlphaViT L4 Multi, AlphaViD L1 Multi, AlphaVDA L1 Multi) remain stable without a pronounced decline, highlighting multitask learning as a potentially effective approach for mitigating overfitting.

Small board configurations. Agents trained on smaller board sizes exhibit notably faster convergence, attaining peak Elo ratings within 300 iterations ( $\approx 100$ for Connect 4 5 × 4). After this rapid convergence, in Connect 4 5 × 4 and Gomoku 6 × 6, Elo ratings remain stable with minimal fluctuation, suggesting that small board training reliably and quickly yields robust and stable agents. However, in Othello 6 × 6, Elo ratings exhibit slight fluctuations of approximately $\pm 100$.

Insights and implications. These findings clearly demonstrate that AlphaViT L4, AlphaViD L1, and AlphaVDA L1 quickly achieve high-performance levels on small boards. However, achieving similar performance stability on larger board configurations remains more challenging. The observed plateau in Elo improvement on large boards implies that simply extending training iterations beyond a certain point (around 1,000 iterations) yields limited additional benefit. Therefore, to achieve further performance improvements on larger boards, enhancing model complexity—such as employing deeper transformer encoders—rather than solely prolonging training duration may be necessary.

Effect of transformer encoder depth on performance

In this subsection, the performance of the agents with deeper encoders, namely AlphaViT L8, AlphaViD L5, and AlphaVDA L5, is evaluated. Figure 5 shows the variations in Elo ratings for these agents over the iterations on large and small boards. All experiments followed the evaluation protocol described in the Setup paragraph of ‘Variation of Elo rating over training iterations’.

For large board configurations, the Elo ratings of agents with deeper encoders gradually stabilize between roughly 1,000 and 2,000 iterations. This trend indicates a more protracted improvement phase compared to their shallower counterparts (AlphaViT L4, AlphaViD L1, and AlphaVDA L1). Conversely, in small board configurations, Elo ratings, as well as their 95% confidence intervals, converge more rapidly, similar to those of the shallow encoder agents. This suggests that in less complex game environments, the additional depth of encoders offers little substantial benefit.

Table 6 lists the mean Elo ratings calculated from iterations 2,100–3,000 for large board configurations. This highlights the performance gains achieved through the increased encoder depth, with AlphaViD and AlphaVDA demonstrating the most notable improvements. For example, AlphaViD L5 Multi and AlphaVDA L5 Multi achieve gains of $+286$ and $+231$ Elo points, respectively, in Gomoku, indicating significant performance enhancement.

The ratio of Elo ratings between the deeper and baseline DNNs, illustrated in Fig. 6, further substantiates these findings. Across all evaluated games, deeper architectures generally exhibit superior performance, with AlphaViD Multi and AlphaVDA Multi achieving the highest ratios. Specifically, in Gomoku, these agents achieve Elo ratios of 1.177 and 1.149, respectively. They also outperform the baseline models in Connect 4, with Elo ratios of 1.108 and 1.130, respectively. While single-game-trained AlphaViD and AlphaVDA show marked gains in Connect 4 and Othello, their performance improvements in Gomoku are modest. In contrast, both single- and multi-game-trained AlphaViT variants exhibit relatively smaller improvements than AlphaViD and AlphaVDA. In conclusion, these results collectively demonstrate that increasing the depth of the transformer encoder layers positively influences agent performance across various game types and board configurations. This effect is particularly pronounced for AlphaViD and AlphaVDA, especially in larger and more complex game settings.

Effect of fine-tuning from small board games

Figure 7 illustrates the Elo ratings of AlphaViT L8, AlphaViD L5, and AlphaVDA L5 with fine-tuned and non-fine-tuned (randomly initialized) DNNs across three games: Connect 4, Gomoku, and Othello. The fine-tuned DNNs were initialized using weights from the single-task DNNs that had been trained for 200 iterations on the small board configuration. For example, the fine-tuned DNN for Connect 4 was initialized with weights from the DNN trained on Connect 4 5 × 4. In contrast, agents with non-fine-tuned DNNs were initialized with random weights. All experiments followed the evaluation protocol described in the Setup paragraph of ‘Variation of Elo rating over training iterations’. For agents with non-fine-tuned (randomly initialized) DNNs, the Elo ratings previously reported in ‘Effect of transformer encoder depth on performance’ were reused.

The results demonstrate that agents with fine-tuned DNNs consistently achieve higher Elo ratings than those with randomly initialized DNNs for Gomoku and Othello. The performance difference is especially pronounced during the early iterations; in this phase, fine-tuned DNNs improve rapidly and then stabilize for Gomoku and Othello. For Gomoku, the advantage of fine-tuning is clear, as the agents with fine-tuned DNNs consistently outperform those with non-fine-tuned DNNs throughout the training process. The plateau appears in AlphaViD L5 and AlphaVDA L5 for Gomoku between approximately 1,000 and 2,000 iterations. However, the performance improves after 2,000 iterations. For Connect 4, fine-tuning yields little or no improvement in Elo.

AlphaZero

AlphaZero integrates a deep neural network (DNN) with Monte Carlo Tree Search (MCTS), as illustrated in Fig. 1. The DNN processes input representing the current board state and the current player, producing an estimated state value and a move-probability vector. MCTS then uses these outputs to select the optimal move. This same framework is adopted by AlphaViT, AlphaViD, and AlphaVDA.

Deep neural network in AlphaZero

AlphaZero’s DNN predicts a value $v(s)$ and a move-probability vector $\mathit{p}(s)$ with components $p(a\mid s)$ for each action $a$, given a state $s$. In the board game context, $s$ and $a$ represent the board state and the move, respectively. The DNN receives input representing the current board state and the current player’s disc color. Figure A1 illustrates the DNN architecture, which consists of a Body (residual blocks) and two Heads (value and policy heads). The value head outputs the estimated state value $v(s)$, while the policy head produces the move probabilities $\mathit{p}(s)$.

The input to the DNN is an $H\times W\times (2T+1)$ image stack that contains $2T+1$ binary feature planes of size $H\times W$. Here, $H\times W$ refers to the board size, and T is the number of histories (previous board states). The first T feature planes represent the occupancy of the player’s discs, with a feature value of 1 indicating that a disc occupies the corresponding cell, and 0 otherwise. Similarly, the following T feature planes represent the occupancy of the opponent’s discs. The final plane encodes the current player, being filled with $+1$ when it is the first player’s turn and with $-1$ otherwise.

Monte Carlo tree search in AlphaZero

This subsection provides an explanation of the Monte Carlo Tree Search (MCTS) algorithm used in AlphaZero. Each node in the game tree represents a game state, and each edge $(s,a)$ represents a valid action from that state. The edges store a set of statistics: $\{N(s,a),W(s,a),Q(s,a),p(s,a)\}$, where $N(s,a)$ is the visit count, $W(s,a)$ is the cumulative value, $Q(s,a)=W(s,a)/N(s,a)$ is the mean value, and $p(s,a)$ is the move probability.

The MCTS for AlphaZero consists of four steps: Select, Expand and Evaluate, Backup, and Play. A simulation is defined as a sequence of Select, Expand and Evaluate, and Backup steps, repeated $N_{\mathrm{s}\mathrm{i}\mathrm{m}}$ times. Play is executed after $N_{\mathrm{s}\mathrm{i}\mathrm{m}}$ simulations.

In Select, the tree is searched from the root node $s_{\mathrm{r}\mathrm{o}\mathrm{o}\mathrm{t}}$ to the leaf node $s_L$ at time step L using a variant of the Polynomial Upper Confidence Trees (PUCT) algorithm. At each time step $t<L$, the selected action $a_t$ has the maximum score, as described in Eq. (9):

(9) $$a_t=argma{x}_a(Q(s_t,a)+C_{\mathrm{p}\mathrm{u}\mathrm{c}\mathrm{t}}p(s_t,a)\frac{\sqrt{N(s_t)}}{1+N(s_t,a)}),$$where $N(s_t)$ is the number of parent visits and $C_{\mathrm{p}\mathrm{u}\mathrm{c}\mathrm{t}}$ is the exploration rate. In this study, $C_{\mathrm{p}\mathrm{u}\mathrm{c}\mathrm{t}}$ is constant, whereas in the original AlphaZero, $C_{\mathrm{p}\mathrm{u}\mathrm{c}\mathrm{t}}$ increases slowly with search time (Silver et al., 2018). Additionally, when the parent node is the root node, the node selection is performed using an $\epsilon$-greedy algorithm based on the UCT scores, where $\epsilon$ denotes the exploration probability.

In Expand and Evaluate, the DNN evaluates the leaf node and outputs $v(s_L)$ and $\mathit{p}(s_L)$. If the leaf node is a terminal node, $v(s_L)$ is the color of the winning player’s disc. The leaf node is expanded and each edge $(s_L,a)$ is initialized to $\{N(s_L,a)=0,W(s_L,a)=0,Q(s_L,a)=0,p(s_L,a)=p(a\mid s_L)\}$.

In Backup, the visit counts and values are updated for each step $t\le L$ during the backward pass. The visit count is incremented by 1, $N(s_t,a_t)\leftarrow N(s_t,a_t)+1$, and the cumulative and average values are updated, $W(s_t,a_t)\leftarrow W(s_t,a_t)+v$, $Q(s_t,a_t)\leftarrow W(s_t,a_t)/N(s_t,a_t)$.

Finally, in Play, AlphaZero selects the action corresponding to the most visited edge from the root node.