Evaluating machine learning models for predictive accuracy in cryptocurrency price forecasting

Contribution and novelty

This study uniquely contributes by comparing models such as logistic regression, random forest (RF) and gradient boosting within high-volatility settings, addressing data imbalance using techniques like SMOTE for realistic model assessments.

Methodology and data

We utilize historical cryptocurrency data from exchanges and aggregators, preprocessed and resampled to manage class imbalance, and evaluate models through hyperparameter tuning and backtesting to ensure their practical applicability.

Research questions

This article investigates how ML classification models perform in cryptocurrency trading and identifies which models offer the most consistent accuracy and reliability.

Structure of the article

The article proceeds as follows: the Introduction provides context, the Literature Review surveys relevant research, the Methodology section outlines model selection and evaluation processes, Results examine model performance, and the Discussion and Conclusion summarize findings and explore future directions.

This study aims to compare ML-based classification algorithms to determine the most effective for algorithmic trading, offering insights into the effectiveness of different algorithms for securing profitable transactions. Building on work by Nguyen & Ślepaczuk (2022), we evaluate the predictive power of various ML algorithms within cryptocurrency markets, focusing on forecast accuracy over complete trading strategies. While accurate forecasts are essential, future research might incorporate these predictions into trading systems to assess profitability under live market conditions. Through detailed comparative analysis, this study seeks to uncover the most effective methods for predicting cryptocurrency prices and refining algorithmic trading strategies, providing traders with actionable insights for sound investment decisions. The findings have the potential to enhance algorithmic trading frameworks, improving accuracy and profitability within the cryptocurrency market and promoting a more resilient trading environment.

However, this study faces several limitations:

• Crypto market volatility: The high unpredictability of the cryptocurrency market can impact algorithmic trading performance.

• Model generalization: Models effective on historical data may not generalize well in future conditions, affecting predictive reliability.

• Technical indicator selection: Technical indicator selection remains subjective, complicating comparisons across trading algorithms.

• Data quality and availability: Limited access to comprehensive, unbiased historical data may restrict the accuracy of findings.

• In this research, we evaluate classification techniques and technical indicators to forecast algorithmic trading outcomes in cryptocurrency markets, assessing the viability of different models for making investment decisions in volatile conditions. The study provides insights into the strengths and limitations of each approach and explores ways to integrate them for more accurate, profitable trading strategies. This work focuses on a selected timeframe and group of cryptocurrencies, highlighting how historical data, metrics like accuracy and profitability, and limitations can guide future improvements in algorithmic trading within the crypto market.

Efficient market hypothesis and market efficiency

The efficient market hypothesis (EMH) has suggested that markets are efficient, reflecting all available information and challenging consistent forecasting. Ying et al. (2019) examined EMH, indicating that reliable market forecasting may be inherently limited. Almotiri & Al Ghamdi (2022a) extended this, proposing that market trends lead to a self-correcting mechanism, reducing historical data predictability. Despite this, several studies have shown that certain algorithms may still provide useful predictions.

Machine learning for stock market and financial predictions

Several studies apply machine learning to analyze market behavior and enhance trading strategies:

• Return direction prediction: Ghoddusi, Creamer & Rafizadeh (2019) used machine learning on ETF returns, finding that models like SVM and Naive Bayes outperform traditional strategies, underscoring algorithmic trading’s potential.

• Stock market forecasting: Nti, Adekoya & Weyori (2020) applied machine learning to time series analysis for stock prediction, showing that ML models achieve high accuracy despite forecasting complexities.

• High-frequency trading: Almotiri & Al Ghamdi (2022b) highlighted algorithmic trading’s significant role, especially in high-frequency trading, where it dominates equity transactions in the U.S.

Algorithmic trading and strategy implementation

• Automated trading strategies: Jansen (2018) explored Python APIs for automated trading, highlighting the importance of precision and backtesting.

• Advanced ML models: Hartanto, Kholik & Pristyanto (2023) emphasized the accuracy of XGBoost and LightGBM for stock price prediction, helping investors adjust strategies effectively.

• Decision tree models: Basak et al. (2019) described two-layer bias decision trees, offering practical applications for stock trading recommendations.

Machine learning in cryptocurrency markets

Given the unique challenges of cryptocurrency, machine learning has found significant application:

• Portfolio management: Bhutta et al. (2020) showed that ML-driven strategies for portfolio balancing can outperform traditional strategies in volatile markets.

• Cryptocurrency forecasting models: Ren et al. (2022) found that gradient-boosting decision trees, long short-term memory, and random forests perform well for short-term predictions. Similarly, Iqbal et al. (2021) showed that simpler models may outperform complex algorithms for cryptocurrency prices.

• Performance in volatile markets: Abakah et al. (2020) modeled the persistent volatility in cryptocurrencies, suggesting that integrating persistent features can improve prediction accuracy.

Sentiment analysis and investor behavior in cryptocurrency markets

• Sentiment-based models: Wimalagunaratne & Poravi (2018) incorporated social media sentiment into ML models, showing significant improvements in cryptocurrency price prediction accuracy.

• Investor sentiment as a predictor: Zhu et al. (2021) used Google Trends data to represent investor sentiment, showing it as a strong predictor of Bitcoin returns and underscoring the role of sentiment in crypto volatility.

Advanced ML techniques and evaluation in crypto market applications

• Neural networks and autoencoder models: Alsuwat (2023) explored hybrid autoencoders for multi-class price movement prediction in cryptocurrency, achieving stable performance metrics. Darwish et al. (2020) recommended time-persistent retraining, addressing overfitting in financial predictions.

• Deep learning and boosted trees: Oyedele et al. (2023) compared CNNs with boosted trees, finding CNNs to have the highest variance scores, validating the effectiveness of DL models for cryptocurrency prediction.

Risk and volatility in cryptocurrency investments

• Portfolio construction with cryptocurrencies: Aharoni (2015) examined cryptocurrencies’ role in traditional portfolios, finding that assets like Ripple enhance portfolio optimization alongside Bitcoin.

• Volatility persistence and trend trading: Omane-Adjepong, Alagidede & Akosah (2019) investigated price persistence in cryptocurrencies, suggesting that while market efficiency is improving, trend trading remains viable for profit-making

Cryptocurrency market studies

• The cryptocurrency market’s substantial volatility creates challenges for accurate price forecasting, necessitating advanced ML models that can adapt to dynamic shifts (Ahmed et al., 2024). Traditional metrics such as mean absolute error (MAE), mean squared error (MSE), and root mean square error (RMSE) are commonly used to gauge the predictive performance of these models, particularly in high-dimensional datasets (Chowdhury et al., 2020; Mudassir et al., 2020).

• Many existing studies focus on various ML techniques to achieve predictive accuracy, yet these models often encounter issues with high dimensionality and scalability, limiting their adaptability to the unique volatility of cryptocurrency markets. For instance, Grudniewicz & Ślepaczuk (2023) highlight the adaptability of ML models in stock markets, pointing out that challenges with high volatility and overfitting frequently impact the accuracy of cryptocurrency forecasts. The current research addresses these limitations by aiming to improve model robustness to enhance predictive accuracy in cryptocurrency markets.

Machine learning model studies

• Random forest and gradient boosting: RF and gradient boosting are frequently utilized for cryptocurrency price forecasting due to their strong predictive performance with historical data and ability to manage data noise effectively. RF models, in particular, demonstrate minimal RMSE values, positioning them as reliable models for high-frequency trading (Al Ghamdi, 2023; Ahmed et al., 2024). However, these models often struggle with scalability and interpretability, which can hinder their application in real-time trading contexts (Grudniewicz & Ślepaczuk, 2023).

• LSTM and gated recurrent unit (GRU): Deep learning models, particularly LSTM and GRU, show significant advantages in time-series forecasting due to their ability to capture time dependencies, making them effective in predicting short-term cryptocurrency price movements. Studies by Hamayel & Owda (2021) and Gudavalli & Kancherla (2023) confirm that these models outperform traditional methods in sequential forecasting tasks, although they are computationally demanding and require substantial data for training, which limits their real-time utility (Yin et al., 2024a).

• Support vector machine (SVM): While SVM has shown success in classification tasks, it faces limitations in volatile environments like cryptocurrency markets due to its lack of flexibility for sudden market shifts (Zhang, Cai & Wen, 2024). Recent studies suggest hybrid models that incorporate SVM with neural networks as potential solutions to improve predictive performance and adaptability in these environments (Wang et al., 2024).

Algorithmic trading strategies

• Comparative analysis and application in algorithmic investment: Grudniewicz & Ślepaczuk (2023) conducted a thorough comparison of machine learning models applied to quantitative investment strategies, specifically in the global stock market. Their study underscores the adaptability of ML models and the importance of robust, scalable architectures for trading applications. This approach aligns with algorithmic trading needs in cryptocurrency, where high-frequency trading and scalability are essential.

• Scalability and robustness: Studies on algorithmic trading emphasize the necessity of scalable, robust models capable of handling high-frequency data. Ensemble learning methods, such as RF and gradient boosting, achieve high accuracy, but their scalability limitations require enhanced architectures for practical applications in algorithmic trading (Jaquart, Köpke & Weinhardt, 2022). Recent advancements, such as AFBNet and DPAL-BERT, provide solutions by offering more efficient architectures, specifically optimized for dynamic data environments and fast processing (Yin et al., 2024b).

Summary of conclusions from previous research

The review of the literature demonstrates a clear need for machine learning models that balance predictive accuracy with scalability and robustness, particularly in high-volatility markets like cryptocurrency. Despite the success of existing models, challenges remain in managing data dimensionality and sudden market shifts, which limit their application in real-time trading. This study builds on these findings, aiming to optimize machine learning parameters and data-handling techniques to improve accuracy and adaptability in cryptocurrency price forecasting.

A comparative summary of reviewed studies highlights various ML models applied in financial and cryptocurrency market predictions. Ahmed et al. (2024) explored the predictive performance of models such as RF and gradient boosting, finding them highly accurate with historical data but limited in real-time scalability. Grudniewicz & Ślepaczuk (2023) examined ML models for investment strategies, emphasizing their adaptability in stock market environments. However, these models show limited applicability to the cryptocurrency market due to its unique dynamics.

Hamayel & Owda (2021) studied sequential data prediction with LSTM and GRU models, noting their ability to handle time dependencies effectively for short-term predictions, though they are computationally intensive, posing challenges for real-time applications. Yin et al. (2024b) focused on lightweight ML architectures like DPAL-BERT and AFBNet, which excel in rapid processing and efficiency but have limited applications in finance, highlighting the need for more adaptable solutions.

Jaquart, Köpke & Weinhardt (2022) concentrated on algorithmic trading, particularly with ensemble models that provide robustness for high-frequency trading. While these models demonstrate strong performance in managing rapid data flows, they require further scalability enhancements to handle diverse trading environments. This summary illustrates both the strengths and limitations of various ML approaches, underscoring the challenges and areas for improvement in financial and cryptocurrency market applications (Al Hawi, Sharqawi & Al-Haija, 2023).

Robustness analysis

To ensure model reliability, a robustness analysis covering model parameters, data period, and data frequency is essential. Sensitivity analysis assesses how variations in these factors impact model performance, which is critical for applications in dynamic markets. For example, Kijewski, Ślepaczuk & Wysocki (2024) performed sensitivity analysis on ML models predicting S&P500 index prices, demonstrating the importance of assessing parameter stability. Additionally, Zenkova & Ślepaczuk (2018) explored robustness in algorithmic trading applications, providing valuable insights into model resilience in volatile markets. While this study primarily focused on predictive performance, future work may incorporate robustness testing to further validate model effectiveness under varying conditions (Kijewski, Ślepaczuk & Wysocki, 2024; Zenkova & Ślepaczuk, 2018).

The hyperparameter tuning phase utilized methods such as grid search and random search to identify optimal model parameters for each algorithm. Each tuning iteration involved adjusting parameters, such as learning rate, maximum depth, and n-estimators, which were chosen based on their impact on model accuracy and F1-score. This process involved evaluating each configuration on the validation dataset, with the final parameter sets selected based on performance metrics including accuracy, precision, and recall, leading to enhanced empirical performance (Ayub, Ahsan & Qureshi, 2022).

For each machine learning model, final hyperparameters were chosen based on their impact on prediction accuracy and stability. Key hyperparameters included the learning rate, regularization strength, and maximum depth for tree-based models, with specific settings optimized through cross-validation. For example, random forest was optimized with a depth of 10 and 100 estimators, while XGBoost utilized a learning rate of 0.01 and a maximum depth of 5. These parameter values were chosen based on iterative testing to maximize F1-score and minimize error variance, enhancing model predictability under diverse conditions (Friedman, 2001).

Hyperparameter tuning covered a range of values to ensure optimal model performance. For instance, the learning rate was varied from 0.001 to 0.1, the maximum depth for tree-based models ranged from 3 to 15, and the number of estimators spanned 50 to 200. This comprehensive range allowed each model to be tested under various configurations, enhancing its adaptability to market conditions while ensuring robustness against overfitting. The tuning process employed grid search and Bayesian optimization methods to identify combinations that maximized both predictive power and model stability (Ayub, Ahsan & Qureshi, 2022).

To align the model’s performance with the practical realities of algorithmic trading, the standard error metrics were replaced by a return-based loss function, which evaluates model predictions in terms of realized returns. This approach follows methodologies outlined in recent studies (Michańków, Sakowski & Ślepaczuk, 2022), which emphasize aligning forecast evaluation with actual trading outcomes. This transition enables the model to prioritize profit-generating predictions over purely statistical accuracy, ensuring that the model’s decisions align with financially meaningful outcomes (Michańków, Sakowski & Ślepaczuk, 2022).

Data collection

The foundation of any robust algorithmic trading strategy is comprehensive and accurate data. The historical data collection process is crucial in the algorithmic trading of cryptocurrency markets and should cover a sufficient period to enable meaningful analysis and modeling. The essential types of data collected include:

• Prices: Historical price data for the cryptocurrency assets includes the date, time, and closing prices. This information is vital for analyzing price trends and volatility.

• Volume: Trading volume data for each cryptocurrency asset provides insights into its liquidity and market activity. This data is crucial for understanding market dynamics and identifying potential buy or sell signals.

Data description

This study utilizes essential features for cryptocurrency price prediction, including historical price, trading volume, and volatility indicators, selected for their relevance to market dynamics and predictive accuracy.

• Historical price captures price trends and momentum, providing a foundation for forecasting future movements.

• Trading volume reflects market activity and investor interest, offering insights into the strength behind price movements.

• Volatility indicators measure price fluctuations, enabling the model to assess market stability and associated risks.

• These features were selected due to their impact on model accuracy and alignment with study objectives. As Nguyen & Ślepaczuk (2022) note, feature configuration significantly affects model performance, and each input’s contribution was carefully evaluated for its role in enhancing model effectiveness, with future potential to explore alternative configurations.

Data specifics

• 1)

  Frequency: Data was gathered on an hourly basis, providing a granular view of market dynamics essential for detecting patterns in a highly volatile environment.

• 2)

  Data range: The time series covers January 2018 to December 2022, capturing both bullish and bearish market conditions. This comprehensive range enables an in-depth analysis of model performance across different phases of the market.

• 3)

  Rationale for chosen time series: This period was selected as it includes critical phases of growth and decline in the cryptocurrency market, enhancing the robustness and relevance of model training and testing.

• 4)

  Rationale for selected assets: The study focuses on major cryptocurrencies, such as Bitcoin and Ethereum, due to their high liquidity, significant market influence, and established trading patterns. Other assets were not included due to data limitations and lower relevance for this analysis.

• 5)

  Descriptive statistics: Key metrics, including mean, median, standard deviation, minimum, and maximum values for each asset, provide insights into asset volatility and distribution. These statistics are essential for understanding data variability and model requirements.

• 6)

  Data sources: The data was sourced from well-established cryptocurrency exchanges and aggregators known for comprehensive and reliable market data, ensuring data integrity and accuracy in modeling.

• 7)

  Data transformation: Preprocessing involved normalization to ensure feature comparability, resampling with SMOTE to address class imbalance, and feature extraction for indicators such as MACD, EMA, RSI, and Bollinger Bands. These transformations improve model interpretability and enhance predictive performance.

Historical data sources

Several resources are available for collecting cryptocurrency data, including:

• Crypto exchanges: Many cryptocurrency exchanges offer APIs that access historical data.

• Cryptocurrency data aggregators: Websites compiling data from multiple sources can provide a comprehensive market view.

Crypto exchanges

Specifically, data from cryptocurrency exchanges can be accessed via APIs. Here are examples of top-rated exchanges providing historical data:

Cryptocurrency data aggregators

Cryptocurrency data aggregators are platforms that compile, process, and provide cryptocurrency-related data from various sources (Rashid et al., 2021). These platforms are essential tools for traders, investors, and researchers who require access to real-time market information, historical data, and other relevant metrics to inform their trading decisions. Here are some of the well-known cryptocurrency data aggregators:

CoinMarketCap: CoinMarketCap is a leading cryptocurrency data aggregator that offers APIs for accessing a wide range of data. To utilize the CoinMarketCap API:

Register for an API key: Users must create an account on CoinMarketCap and register for an API key, which is necessary to authenticate API requests (Hegnauer, 2019).

Choose an API endpoint: The platform provides various API endpoints to access cryptocurrency prices, trading volumes, and historical data.

Make a request: With the API key and chosen endpoint, users can make requests to retrieve data. This can be done using programming tools like Python or applications like Postman (Brown, Wei & Wermers, 2014).

Parse the response: The API response is typically in JSON format, which can be parsed using a JSON library in Python or visualized using Postman for more straightforward analysis.

Store and analyze the data: Once retrieved, the data can be stored and analyzed to derive insights about the cryptocurrency market.

Note: CoinMarketCap imposes rate limits on API usage, so users are advised to be mindful of their request frequency to avoid hitting these limits.

CryptoSlate: CryptoSlate is another resource that offers APIs for data retrieval (Shivsharan et al., 2023):

Register for an API key: Similar to CoinMarketCap, users must register for an API key on CryptoSlate to access their APIs.

Choose an API endpoint: CryptoSlate provides several endpoints to access diverse data types.

Make a request: API requests can be made using a programming language like Python or tools such as Postman.

Parse the response: Responses are provided in JSON format, which can be parsed for further analysis.

CoinGecko: CoinGecko offers a comprehensive API similar to the ones provided by CoinMarketCap and CryptoSlate. The steps for using the CoinGecko API are analogous to those described for CryptoSlate (Amsden & Schweizer, 2018).

Nomics: Nomics provides APIs for data retrieval with steps for usage similar to those of CryptoSlate. Their API allows developers to access extensive cryptocurrency data. As shown in Fig. 1, the candlesticks pattern highlights the volatility trends during backtesting.

Tools and technologies

Several tools and technologies were utilized to conduct this research effectively, enabling efficient data handling, model evaluation, and backtesting. Below is a detailed explanation of the tools and technologies used:

Environment setup

To ensure a robust and reliable analysis, the following environment setup steps were followed:

• 1)

  Install Python: Installation of the latest Python version to support all dependencies.

• 2)

  Install required libraries: Installation of necessary Python libraries such as Pandas, NumPy, Scikit-learn, and others via pip.

• 3)

  Data acquisition: Gathering data from various sources, such as cryptocurrency exchanges and data aggregators, ensuring thorough cleaning and preprocessing to maintain data quality.

• 4)

  Development environment setup: Configuration of a local or cloud-based development environment with necessary IDEs and version control systems.

• 5)

  Backtesting and evaluation: Comprehensive backtesting of developed trading strategies using historical data to assess their efficacy and modify them as necessary.

Model selection and training

The model selection process in this research involved evaluating various classification models to identify those most effective in predicting cryptocurrency prices. Selection was based on performance metrics derived from a validation set, with iterative adjustments to models and parameters to optimize results. Initially, models were trained using data splits of 70/30% and 80/20%. However, early outcomes revealed performance issues, largely due to data imbalance. To address this, re-sampling techniques like SMOTE were applied, resulting in improved model training outcomes. For ML models applied in algorithmic trading, the choice of loss function is critical to aligning model optimization with trading profitability goals. While traditional loss functions, such as MSE, focus on minimizing prediction errors, they may not effectively capture directional accuracy, which is vital for trading. Mean absolute directional loss (MADL), as suggested by Michańków, Sakowski & Ślepaczuk (2023), may enhance directional forecasting by directly penalizing directional errors. Adopting such a loss function could improve the models’ predictive relevance in trading contexts, presenting a valuable consideration for future research (Chen, Li & Sun, 2020).

Balancing the dataset

Balancing the dataset is crucial in machine learning, particularly in classification problems. Imbalanced datasets can cause models to be biased, leading to unreliable predictions. This section explores the concepts and techniques used to achieve balance in datasets, focusing on the majority and minority classes and the re-sampling methods employed.

Under-sampling

Under-sampling targets reducing the number of instances from the majority class to equalize the class distribution (Arafat, Hoque & Farid, 2017). This method is straightforward and can quickly help balance the class ratio; however, it risks losing important information that could be crucial for model learning. Therefore, while under-sampling can be effective, especially in large datasets, it should be employed cautiously. Combining under-sampling with oversampling methods or advanced algorithms like SMOTE can provide a more balanced approach and better performance outcomes. Figure 2 shows an image of an under-sampling.

Oversampling

One of the critical approaches of oversampling and common cases in data preprocessing, mainly if the minority class rarely occurs, is a data preprocessing technique employed to tackle the class imbalance problem Sklearn. This procedure aims to boost the model’s prediction quality in problems with class distribution imbalance. Therefore, it duplicates the minority data, resulting in a similar number of instances as the majority class. As a result, it may be performed using duplication, but the synthesized instances appear with the help of generating new ones. One of the standard methods to create new instances effectively is called SMOTE, which stands for Synthetic Minority Over-sampling Technique (Sajid et al., 2023). Figure 3 shows an image of over-sampling.

After model training

However, the training of models completed with the initial data splits in this study gave inferior results, according to the concept of imbalanced data. SMOTE was, however, applied as a re-sampling technique to enhance better performance for the model. However, after resampling, the improvement was still not visible, which led to the decision to hyperparameter-tune the listed models for better performance optimization. This further hyperparameter tuning becomes of high importance, as it tunes the settings of the model for better capture of the details in the data, especially after the change of class distributions, which can change the dynamics of how models learn from the data. Figure 4 shows an image of the smote steps.

Hyperparameter

Hyperparameter tuning is a critical process for optimizing machine learning model performance (Ayub, Ahsan & Qureshi, 2022). Unlike model parameters learned during training, hyperparameters are preset before training and are determined through experience or experimentation. These include factors such as the number of trees in a random forest, the layers in a neural network, and the regularization term in a regression model, all of which shape model behavior and learning capability. The goal of hyperparameter tuning is to identify the combination that yields the best performance for a specific task. Common methods include grid search, random search, and Bayesian optimization, each offering a systematic way of exploring the hyperparameter space.

After selecting an optimal set of hyperparameters, each model was retrained using these values, tested with 70/30 and 80/20 data splits to determine which configuration produced higher performance. Optimizing hyperparameters enhances model alignment with data characteristics, reducing issues of underfitting or overfitting and leading to performance gains.

In this study, key hyperparameters—such as learning rate, batch size, and the number of layers—were chosen based on relevance to each model and tuned over a historical period reflective of the dataset’s characteristics. Parameter ranges were defined from initial testing, and a grid search approach was used to systematically explore combinations. This structured tuning process aligns with best practices in financial model optimization, as demonstrated by Wysocki & Ślepaczuk (2021), who used a structured approach to tune neural networks in financial applications (Wysocki & Ślepaczuk, 2021). Future work may explore random search to further refine parameter selection.

Backtesting methodology

Backtesting is a critical component of algorithmic trading, allowing for the evaluation of trading strategies or models using historical market data. This process simulates trade execution based on predefined rules or models, thereby assessing performance under historical market conditions. In our study—focused on comparing classification models and technical indicators in the context of cryptocurrency market data—backtesting plays a central role in validating the practical effectiveness of these tools.

By examining past performance, backtesting enables a clear assessment of each model’s strengths and weaknesses, providing traders with valuable insights into when and how to deploy these models and indicators in real trading environments. This empirical approach, grounded in observed rather than theoretical performance, empowers traders to fine-tune their strategies for optimal outcomes.

For an in-depth view, Table 1 presents an analysis of equity lines, detailing each model’s cumulative return, volatility, and Sortino Ratio, which highlight risk-adjusted performance. Additionally, Table 2 displays the statistical significance of each model’s performance metrics, validated through p-values and confidence intervals, ensuring robust performance comparison across models.

Dataset

The dataset employed in this research was sourced from three distinct files to provide a comprehensive view of market dynamics on the Binance exchange:

• Bought.filter: This file contains details of the coins purchased on Binance over a specified period. It is instrumental in understanding the buying patterns and preferences of traders, which are crucial for analysing market trends and predicting future movements.

• Sold.Filter: Although not detailed in the outline, a corresponding file typically contains data on coins sold during the same timeframe, providing a mirror image of buying behavior and further insights into market exits and trader strategies.

• Historical Data: This includes 1 month of historical data for all coins traded on Binance. The comprehensive nature of this data allows for in-depth analysis and modeling, offering a broad perspective of market behaviors over the period.

Structure of bought file

The “Bought.filter” file is structured to capture essential details about transactions, including the time of purchase, the type of coin bought, the quantity, and the price at which the purchase was made. Analysing this data helps in understanding the timing and scale of entries into various cryptocurrency positions. Insights from this file are pivotal for developing strategies that align with observed market entry trends and for anticipating future market movements based on historical buying patterns. By dissecting the structure and content of this dataset, the research aims to extract meaningful patterns and indicators that can drive successful algorithmic trading strategies in the volatile cryptocurrency market (Qureshi et al., 2024). The subsequent sections will delve into how these datasets were utilized to train and refine the predictive models, leading up to a thorough evaluation through backtesting to confirm the strategies’ effectiveness in real-world trading scenarios. The area chart shows in Fig. 5 the coin distribution for different strategies with for each strategy plotted on the y-axis and the associated coins listed on the x-axis. The chart represents graphically how the coins are distributed among various strategies. The bubble chart in Fig. 6 displays the distribution of coins across all strategies, with the x-axis representing the list of coins and the y-axis showing the percentage of coins allocated to each strategy. Figure 6 also illustrates the distribution of trading strategies, providing a visual comparison between the CrossOver and MA44 strategies. Table 3 shows the content of Bought.filter.

Table 3:

Trading strategy execution for selected cryptocurrency pairs.

This table details the Date, Time, Coin, and Strategy used in executing trades for selected cryptocurrency pairs. Date and Time specify when each trade was executed. Coin lists the traded cryptocurrency pairs. Strategy indicates whether the CrossOver or MA44 strategy was applied. This table serves as a trading log, providing insight into the strategies employed at specific times, which is valuable for analyzing the performance of trading strategies in real-time market conditions.

Date	Time	Coin	Strategy
2024-06-15	03:30:00	TROYBUSD	CrossOver
2024-06-15	04:30:00	SANTOSBUSD	MA44
2024-06-15	04:30:00	COCOSBUSD	MA44
2024-06-15	05:00:00	BURGERBUSD	CrossOver
2024-06-15	05:00:00	PONDBUSD	CrossOver
2024-06-15	05:30:00	INJBUSD	CrossOver
2024-06-15	05:30:00	ELFBUSD	MA44
2024-06-15	06:00:00	ALPINEBUSD	MA44
2024-06-15	06:00:00	CLVUSD	MA44
2024-06-15	06:30:00	ATOMBUSD	CrossOver

DOI: 10.7717/peerj-cs.2626/table-3

Structure of sold file

The “Sold.filter” data source contains data that pertains to coins that have been sold through transactions over Binance exchanges. Figure 7 presents the distribution of coins along the x-axis and their corresponding gains on the y-axis, with each cell color-coded to indicate the level of gain. This visualization aids in identifying the most profitable strategies, as different colors or lines represent various trading strategies. Whereas Table 4 shows data for sold filter.

Table 4:

Trading gains for selected cryptocurrency pairs using specific strategies.

This table lists the Date, Time, Coin, and Gain obtained from trading specific cryptocurrency pairs using designated strategies. Gain represents the profit or loss from each trade. This summary allows for a performance analysis of each trading strategy, highlighting the effectivene­­ss of different approaches across various cryptocurrency pairs over time.

Date	Time	Coin	Gain
2024-06-15	03:30:00	TROYBUSD	1.70820498
2024-06-15	04:30:00	SANTOSBUSD	0.79230333
2024-06-15	04:30:00	COCOSBUSD	1.69249285
2024-06-15	05:00:00	BURGERBUSD	0.80889787
2024-06-15	05:00:00	PONDBUSD	1.35135135
2024-06-15	05:30:00	INJBUSD	0.92276144
2024-06-15	05:30:00	ELFBUSD	0.87378640
2024-06-15	06:00:00	ALPINEBUSD	3.46534653
2024-06-15	06:00:00	CLVBUSD	0.76489533
2024-06-15	06:30:00	ATOMBUSD	1.62634408

DOI: 10.7717/peerj-cs.2626/table-4

Structure of IDEXBUSD.csv

The IDEXBUSD.csv file provides transaction data at 30-min intervals, with columns for Timestamp, Open Price, High Price, Low Price, Closing Price, and Trading Volume, as elaborated in Table 5. The chart of IDEXBUSD from Fig. 8 shows a smooth uptrend from the starting period to the ending one. Figure 9 is an overlay of the line chart of the IDEXBUSD coin over the historical price data alongside three technical indicators: Bollinger bands, RSI, and SMA. The Bollinger bands show the volatility of the coin price, RSI shows if the coin is overbought or oversold, and SMA points at the direction of the trend.

Table 5:

OHLCV data for cryptocurrency trading.

This table provides the Open, High, Low, Close, and Volume (OHLCV) data for cryptocurrency trading over specified timestamps. Timestamp indicates the exact time each record was captured. Open, High, Low, Close show the price points within a trading interval. Volume represents the trading volume within that interval. This data is crucial for technical analysis, allowing traders to evaluate price trends and make informed decisions based on historical trading activity.

Timestamp	Open	High	Low	Close	Volume
1655184600000	0.0549400	0.0555300	0.0543700	0.0553400	959,931.600
1655186400000	0.0551600	0.0557300	0.0546100	0.0553200	714,559.300
1655188200000	0.0554500	0.0559400	0.0548900	0.0556500	540,882.000
1655190000000	0.0548700	0.0563000	0.0547900	0.0556300	692,693.900
1655191800000	0.0556800	0.0559400	0.0547400	0.0553700	472,159.900
1655193600000	0.0555600	0.0555600	0.0549700	0.0550100	247,727.900
1655195400000	0.0550600	0.0553500	0.0546300	0.0547600	219,821.200
1655197200000	0.0547700	0.0547700	0.0541900	0.0541900	288,802.500
1655199000000	0.0542300	0.0543900	0.0535600	0.0535600	374,959.000
1655200800000	0.0535400	0.0539400	0.0532200	0.0537900	236,314.400

DOI: 10.7717/peerj-cs.2626/table-5

Trained model evaluation

The models were trained on a newly created dataset comprising technical indicators and their corresponding values for a specific cryptocurrency, sourced from various data providers. Figure 9 illustrates the historical data representation for the IDEXBUSD coin, while Fig. 10 visualizes the different technical indicators applied to IDEXBUSD. Table 6 provides additional metrics on model accuracy, recall, and F1-score, supporting performance comparison.

Additionally, Table 7 summarizes the financial performance of each trained model, covering key risk-adjusted metrics such as Sharpe ratio, maximum drawdown, annualized return, and Alpha. These metrics provide a comprehensive comparison of each model’s profitability and risk management capabilities within an algorithmic trading framework.

Model evaluation on 70/30 ratio

Figure 10 bar chart illustrates the accuracy, F1-score, and recall of various models with a 70/30 ratio. The bars are color-coded, with orange representing accuracy, green representing F1-score, and blue representing recall. The y-axis ranges from 0 to 2 displaying the metrics while the x-axis denotes the names of each model. Figure 11 features a scatter chart displaying the accuracy (orange), F1-score (green), and recall (blue) for different models. The y-axis spans from 0 to 1, while the x-axis denotes the model names. The three metrics are represented by orange, green, and blue lines respectively, enabling a comparative analysis across all models. Following the analysis, certain models exhibit superior performance compared to others. The Ridge Classifier, SVC, HistGradientBoost, and XGBoost emerge as the top-performing models in terms of F1-score. Among them the Ridge Classifier boasts the highest F1-score of all models.

Model evaluation on 80/20 ratio

Table 8 displays the evaluation results for the selected models, assessed with an 80/20 data split ratio. Figure 12 presents a bar chart illustrating the accuracy, F1-score, and recall for each model, with orange representing accuracy, green for F1-score, and blue for recall. The y-axis ranges from 0 to 2, while the x-axis lists the model names. Among the best performers with F1-scores above 0.75 are the random forest, gradient boosting, and Ridge Classifier models. SVC and LDA also perform strongly, achieving F1-scores of 0.78 and 0.76, respectively. Figure 9 offers a scatter plot displaying accuracy (orange), F1-score (green), and recall (blue) across models, with the y-axis scaled from 0 to 1. This visual provides a comprehensive comparison of these metrics, further highlighting model performance.

Table 8:

Model evaluation metrics for accuracy, recall, and F1-score.

This table compares machine learning models based on Accuracy, Recall, and F1-Score. Accuracy represents the overall correctness of predictions. Recall shows sensitivity to true positives. F1-Score balances precision and recall. Models like Random Forest, Gradient Boosting, and Support Vector Classifier (SVC) are evaluated, providing a comprehensive view of model performance across key metrics, aiding in selecting models for optimal predictive accuracy and sensitivity.

Model	Accuracy	Recall	F1Score
Logistic regression	0.57	0.57	0.63
SVC	0.64	0.64	0.78
SGD	0.45	0.45	0.35
Gaussian naive bayes	0.36	0.36	0.28
KNeighborsClassifier	0.61	0.61	0.73
Decision tree	0.59	0.59	0.69
Extra tree	0.61	0.61	0.71
Random forest	0.66	0.66	0.76
Bagging	0.61	0.61	0.70
Gradient boosting	0.65	0.65	0.76
AdaBoost	0.63	0.74	0.62
HistGradientBoost	0.62	0.62	0.74
XGBoost	0.64	0.64	0.75
Ridge classifier	0.64	0.64	0.77
Linear Discriminant Analysis (LDA)	0.63	0.63	0.76
Quadratic Discriminant Analysis (QDA)	0.51	0.51	0.50

DOI: 10.7717/peerj-cs.2626/table-8

Additionally, Figs. 11 and 12 depict bar and line charts, respectively, showing model performance with a 70/30 data split, while Fig. 13 provide similar visualizations for the 80/20 split. These figures aid in comparing model performance metrics effectively across different data configurations. The current results give an initial view of the models’ predictive capabilities; however, a full evaluation of their potential in algorithmic trading requires assessing the profitability and robustness of buy/sell signals generated by these forecasts. Future research should consider implementing a structured testing framework with cumulative returns or equity lines as performance indicators, supported by statistical tests, such as t-tests or the Hurst Exponent, to validate the significance of the findings (e.g., Bui & Ślepaczuk, 2022).

Model evaluation on 70/30 ratio

The results for the listed models are shown in Table 9 after balancing the dataset and testing them with a 70/30 split ratio. The stacked bar chart in Fig. 14 displays the accuracy, F1-score, and recall of various models trained on a resampled dataset with a 70/30 ratio. The bars are color-coded with orange for accuracy, green for F1-score, and blue for recall. The y-axis ranges from 0 to 2, while the x-axis lists each model’s name. From the results, it can be proven that many models are highly affected by the under-sampling technique. Logistic Regression and SVC had very low F1-scores of 0.0 and 0.01 respectively. This means those models cannot be used to trade digital currency under the effect of under-sampled data. The best models for F1-score are Random Forest, XGBoost, and HistGradientBoost, with scores all higher than 0.75. These models appear to be relatively robust to under-sampling. Figure 15 we have bar chart the accuracy (orange), F1-score (green), and recall (blue) of the different models. The y-axis ranges from 0 to 1, and the x-axis shows the different model names. The lines are colored in orange for accuracy, green for F1-score, and blue for recall so every detail about the model performance can be known at a glance. Table 9 shows a balanced data evaluation of different models with a 70/30 data. Whereas Fig. 16 shows after re-sampling, the bar chart displays the performance of different models using a 70:30 split ratio. Figure 14 shows image of the line chart shows the performance of several models with a 70:30.

Table 9:

Model performance comparison (accuracy, recall, F1-score).

This table displays the accuracy, recall, and F1-score for various machine learning models, reflecting their performance in classification tasks. Accuracy measures the proportion of correctly classified instances. Recall indicates the model’s ability to detect true positives. F1-score balances precision and recall, providing a composite metric of model performance. Models such as random forest, XGBoost, and gradient boosting are included, allowing for a comparative assessment of their effectiveness in predictive modeling.

Model	Accuracy	Recall	F1Score
Logistic regression	0.49	0.00	0.00
SVC	0.49	0.00	0.01
SGD	0.51	0.35	0.42
Gaussian naive bayes	0.51	1.00	0.68
KNeighborsClassifier	0.59	0.54	0.57
Decision tree	0.72	0.63	0.70
Extra tree	0.72	0.62	0.70
Random forest	0.78	0.76	0.78
Bagging	0.70	0.58	0.67
Gradient boosting	0.71	0.66	0.70
AdaBoost	0.62	0.59	0.61
HistGradientBoost	0.75	0.71	0.75
XGBoost	0.76	0.72	0.75
Ridge classifier	0.60	0.53	0.57
Linear Discriminant Analysis (LDA)	0.59	0.55	0.58
Quadratic Discriminant Analysis (QDA)	0.53	0.99	0.68

DOI: 10.7717/peerj-cs.2626/table-9

Model evaluation on 80/20 ratio

Table 10 presents the assessment results after balancing the dataset and testing with an 80/20 split ratio. The scatter chart in Fig. 17 illustrates the differences in the accuracy, F1-score, and recall of various models trained on the resampled dataset with an 80/20 ratio. The color coding of the bars includes orange for accuracy, green for F1-score, and blue for recall. The y-axis ranges from 0 to 2, and the x-axis lists the different models. Figure 18 bar chart that shows the accuracy (orange), F1-score (green), and recall (blue) of different models. The y-axis scales between 0 to 1, and the x-axis lists the model names. Different colored lines for each metric make it easy to compare all models. The results indicate that imbalanced data affects some models more than others. For example, logistic regression, SVC, and Gaussian naive Bayes exhibit very low F1-scores of 0.0 or 0.01, suggesting these models are not suitable for trading digital currencies with imbalanced data. The top-performing models in terms of F1-score are random forest, HistGradientBoost, and XGBoost, all with F1-scores over 0.77. These models appear relatively resistant to imbalanced data, warranting further analysis. Under-sampling may not always be the best approach for handling imbalanced datasets. Other techniques, such as over-sampling, hybrid-sampling, and SMOTE, might be more effective in certain situations. It is recommended to evaluate the performance of these models using different sampling approaches to draw conclusive inferences under various conditions. Table 7 shows the balanced data evaluation of different models with 80/20 data split. Whereas Fig. 15 shows image of bar chart shows the performance of several models with an 80:20. Figure 17 shows image of line chart shows the performance of several models with an 80:20. Table 11 shows the balanced data evaluation of different parameterized models with 70/30 data split ratio.

Table 10:

Model performance metrics on imbalanced data.

This table provides accuracy, recall, and F1-score metrics for different machine learning models tested on imbalanced data. Accuracy indicates the percentage of correct predictions. Recall measures the ability to identify actual positives in imbalanced scenarios. F1-score assesses model balance between precision and recall. The table compares models like decision tree, random forest, and AdaBoost, offering insights into which models maintain performance robustness when facing data imbalance challenges.

Model	Accuracy	Recall	F1Score
Logistic regression	0.48	0.00	0.00
SVC	0.48	0.00	0.01
SGD	0.48	0.20	0.28
Gaussian naive bayes	0.48	1.00	0.01
KNeighborsClassifier	0.62	0.54	0.59
Decision tree	0.71	0.62	0.69
Extra tree	0.76	0.68	0.74
Random forest	0.80	0.76	0.80
Bagging	0.70	0.60	0.67
Gradient boosting	0.72	0.67	0.71
AdaBoost	0.63	0.62	0.64
HistGradientBoost	0.78	0.75	0.78
XGBoost	0.77	0.72	0.77
Ridge classifier	0.61	0.50	0.57
Linear Discriminant Analysis (LDA)	0.61	0.50	0.57
Quadratic Discriminant Analysis (QDA)	0.55	0.99	0.69

DOI: 10.7717/peerj-cs.2626/table-10

Table 11:

Performance metrics for a selection of machine learning models.

This table shows the accuracy, recall, and F1-score metrics for a selection of machine learning models. Accuracy represents the overall correctness of predictions. Recall reflects the model’s sensitivity to true positives. F1-score combines precision and recall to provide a balanced view of performance. This comparison includes models like logistic regression, SVC, and XGBoost, supporting the evaluation of models for balanced and effective classification performance in trading predictions.

Model	Accuracy	Recall	F1Score
Logistic regression	0.65	1.00	0.79
SVC	0.65	1.00	0.79
KNeighborsClassifier	0.57	0.79	0.70
Decision tree	0.65	1.00	0.79
Extra tree	0.65	1.00	0.79
Random forest	0.64	0.99	0.78
Bagging	0.65	1.00	0.79
AdaBoost	0.65	1.00	0.79
HistGradientBoost	0.63	0.88	0.76
XGBoost	0.65	1.00	0.79

DOI: 10.7717/peerj-cs.2626/table-11

Evaluation of parameterized models

All of the information underwent deep analysis in an attempt to estimate the efficiency of various models in testing hyper-parameter models. Accuracies were tested under very many metrics: recall, F1 score, and many others. The objective here was to identify such models that indicate high levels of accuracy, stability, and robustness in the field of algorithmic trading. In fact, the results were used to deduce some conclusions that could be referred to in offering some recommendations on possible future research by involving them in the development of machine learning-based algorithmic trading.

Parameterized model evaluation on a 70/30 ratio

Evaluation results for the following models after balancing the dataset and testing with a 70/30 ratio split are shown in Table 8 below. A scatter chart of accuracy, F1-score, and recall for different parameterized models at a ratio of 70/30 is shown in Fig. 19. Figure 20 is a bar chart which depicts the accuracy (in orange), F1-score (in green), and recall (in blue) of various models; the y-axis runs up to 1, and the x-axis indicates the model’s name. The corresponding metrics are shown by orange, green, and blue bars; respectively, it makes up a detailed comparison for all the models. From this analysis, it can be observed that some models have really improved in the performance measure after the process of hyperparameter tuning. For example, logistic regression, SVC, decision tree, extra tree, Bagging, and AdaBoost have a very high F1-score of 0.79, while KNeighbors Classifier and XGBoost have a good F1-score of 0.70 and 0.79 respectively. In contrast some models seem to lose their effectiveness after hyperparameter tuning. Among them are Random Forest and HistGradientBoost. This suggests that the improvement of model performance may not be even with hyperparameter tuning and should be carefully assessed and selected.

In general, the results provided rather informative aspects regarding the effectiveness of different machine learning models after hyperparameter tuning for trading digital currencies. It is suggested that more detailed assessment of those models using other methods, for instance, cross-validation, be carried out to gain more valid and stable results. This makes it even more essential that factors like model interpretability, computational resources, and implementation constraints are accounted for in the training of machine learning models for trading in digital currencies. Figure 18 shows image of bar chart displaying the performance of several parameterized models using a 70:30 split ratio.

Parameterized model evaluation on an 80/20 ratio

The results for the listed models of assessment with a balanced dataset and tested with an 80/20 split ratio are as illustrated in Table 12 below. Figure 21 is a scatter chart showing accuracy, F1-score, and recall from different parameterized models with an 80/20 ratio. The color of the lines is color-coded, blue indicating accuracy, orange F1-score, and green recall. It could be plotted in the y-axis, ranging from 0 to 1, and in the x-axis, with the name of the model. Furthermore, we need to take into consideration that the performance of the individual models may be different over various datasets, time series, and market conditions. Therefore, it is recommended that more checks on the models be performed, such as cross-validation, backtesting, or out-of-sample testing, to produce even more reliable and robust results. Among other very important considerations when training a machine learning model for cryptocurrency trading are model interpretability, computational resources, and implementation constraints.

Table 12:

On balanced data evaluation of different parameterised models with 80/20 data split ratio.

This table displays the accuracy, recall, and F1-score metrics for various machine learning models. Accuracy represents the overall proportion of correct predictions. Recall indicates the model’s effectiveness in identifying true positives. F1-score combines precision and recall to provide a balanced performance assessment. The table includes models such as logistic regression, SVC, decision tree, and XGBoost, with most models achieving high recall and F1-scores, highlighting their effectiveness in handling classification tasks. This comparison supports the selection of models that achieve balanced accuracy and sensitivity in predictive applications.

Model	Accuracy	Recall	F1Score
Logistic regression	0.70	0.84	0.76
SVC	0.70	1.00	0.82
KNeighborsClassifier	0.63	0.80	0.75
Decision tree	0.70	1.00	0.82
Extra tree	0.70	1.00	0.82
Random forest	0.69	0.97	0.81
Bagging	0.70	1.00	0.82
AdaBoost	0.70	1.00	0.82
HistGradientBoost	0.63	0.88	0.75
XGBoost	0.70	1.00	0.82

DOI: 10.7717/peerj-cs.2626/table-12

Overall, this result elaborates the performance of different machine learning models put on use in digital currency trade. It would be helpful to run further alternative models and approaches to be more robust and results reliable. The below-shown scatter chart of Fig. 21 represents the values of accuracy (blue), F1-score (orange), and recall (green) for the different models. The y-axis is in the range 0 to 1, and the names of the models are on the x-axis. Different metrics are plotted using different color lines, thus allowing one to examine all three metrics for all models at one go. Figure 19 shows image of chart outcomes of several parameterized models using a 70:30 split ratio. Whereas Table 9 shows the balanced data evaluation of different parameterized models with 80/20 data split ratio. Also Fig. 20 shows image of bar chart displaying the outcomes of several parameterized models using an 80:20 split ratio.

Backtesting results using different ratios

Backtesting is also a process through which the operation of a given investment or trading strategy is applied to historical market data to estimate the performance of such a strategy. In this section, we focus on the evaluation of trading strategies based on data splits of 70/30 and 80/20 ratios.

The 70/30 ratio was used to reserve 70% of the data for training and 30% for testing. This provided insight into the ability of models to generalize to unseen data, simulating real-world scenarios. Similarly, the 80/20 split allowed for the testing of models with a larger training dataset. Both approaches helped understand the potential risk and return in the strategies, aiding in making well-informed decisions before implementing these strategies in live trading environments.

Traders utilize statistical models, mathematical algorithms, and proprietary software programs to run backtests under different configurations. The results show that model performance can vary significantly depending on the data split ratio used. For instance, Fig. 21 shows a chart displaying the evaluation of several parameterized models using an 80:20 split ratio, highlighting which models are more robust under different conditions.