PeerJ Computer Science Preprints: Artificial Intelligencehttps://peerj.com/preprints/index.atom?journal=cs&subject=8300Artificial Intelligence articles published in PeerJ Computer Science PreprintsStreaming stochastic variational Bayes: An improved approach for inference with concept drifting data streamshttps://peerj.com/preprints/277902019-11-192019-11-19Nadheesh JihanMalith JayasingheSrinath Perera
Online learning is an essential tool for predictive analysis based on continuous, endless data streams. Adopting Bayesian inference for online settings allows hierarchical modeling while representing the uncertainty of model parameters. Existing online inference techniques are motivated by either the traditional Bayesian updating or the stochastic optimizations. However, traditional Bayesian updating suffers from overconfident posteriors, where posterior variance becomes too inadequate to adapt to new changes to the posterior with concept drifting data streams. On the other hand, stochastic optimization of variational objective demands exhausting additional analysis to optimize a hyperparameter that controls the posterior variance. In this paper, we present "Streaming Stochastic Variational Bayes" (SSVB) — a novel online approximation inference framework for data streaming to address the aforementioned shortcomings of the current state-of-the-art. SSVB adjusts its posterior variance duly without any user-specified hyperparameters to control the posterior variance while efficiently accommodating the drifting patterns to the posteriors. Moreover, SSVB can be easily adopted by practitioners for a wide range of models (i.e. simple regression models to complex hierarchical models) with little additional analysis. We demonstrate the superior performance of SSVB against Population Variational Inference (PVI), Stochastic Variational Inference (SVI) and Black-box Streaming Variational Bayes (BB-SVB) using two non-conjugate probabilistic models: multinomial logistic regression and linear mixed effect model. Furthermore, we also emphasize the significant accuracy gain with SSVB based inference against conventional online learning models for each task.
Online learning is an essential tool for predictive analysis based on continuous, endless data streams. Adopting Bayesian inference for online settings allows hierarchical modeling while representing the uncertainty of model parameters. Existing online inference techniques are motivated by either the traditional Bayesian updating or the stochastic optimizations. However, traditional Bayesian updating suffers from overconfident posteriors, where posterior variance becomes too inadequate to adapt to new changes to the posterior with concept drifting data streams. On the other hand, stochastic optimization of variational objective demands exhausting additional analysis to optimize a hyperparameter that controls the posterior variance. In this paper, we present "Streaming Stochastic Variational Bayes" (SSVB) — a novel online approximation inference framework for data streaming to address the aforementioned shortcomings of the current state-of-the-art. SSVB adjusts its posterior variance duly without any user-specified hyperparameters to control the posterior variance while efficiently accommodating the drifting patterns to the posteriors. Moreover, SSVB can be easily adopted by practitioners for a wide range of models (i.e. simple regression models to complex hierarchical models) with little additional analysis. We demonstrate the superior performance of SSVB against Population Variational Inference (PVI), Stochastic Variational Inference (SVI) and Black-box Streaming Variational Bayes (BB-SVB) using two non-conjugate probabilistic models: multinomial logistic regression and linear mixed effect model. Furthermore, we also emphasize the significant accuracy gain with SSVB based inference against conventional online learning models for each task.Time series event correlation with DTW and Hierarchical Clustering methodshttps://peerj.com/preprints/279592019-09-122019-09-12Srishti MishraZohair ShafiSantanu Pathak
Data driven decision making is becoming increasingly an important aspect for successful business execution. More and more organizations are moving towards taking informed decisions based on the data that they are generating. Most of this data are in temporal format - time series data. Effective analysis across time series data sets, in an efficient and quick manner is a challenge. The most interesting and valuable part of such analysis is to generate insights on correlation and causation across multiple time series data sets. This paper looks at methods that can be used to analyze such data sets and gain useful insights from it, primarily in the form of correlation and causation analysis. This paper focuses on two methods for doing so, Two Sample Test with Dynamic Time Warping and Hierarchical Clustering and looks at how the results returned from both can be used to gain a better understanding of the data. Moreover, the methods used are meant to work with any data set, regardless of the subject domain and idiosyncrasies of the data set, primarily, a data agnostic approach.
Data driven decision making is becoming increasingly an important aspect for successful business execution. More and more organizations are moving towards taking informed decisions based on the data that they are generating. Most of this data are in temporal format - time series data. Effective analysis across time series data sets, in an efficient and quick manner is a challenge. The most interesting and valuable part of such analysis is to generate insights on correlation and causation across multiple time series data sets. This paper looks at methods that can be used to analyze such data sets and gain useful insights from it, primarily in the form of correlation and causation analysis. This paper focuses on two methods for doing so, Two Sample Test with Dynamic Time Warping and Hierarchical Clustering and looks at how the results returned from both can be used to gain a better understanding of the data. Moreover, the methods used are meant to work with any data set, regardless of the subject domain and idiosyncrasies of the data set, primarily, a data agnostic approach.Mice tracking using the YOLO algorithmhttps://peerj.com/preprints/278802019-08-012019-08-01Helton Maia PeixotoRichardson Santiago TelesJohn Victor Alves LuizAron Miranda Henriques-AlvesRossana Moreno Santa Cruz
The development of computational tools is essential for the development of new technologies, including experimental designs needed for behavioral neuroscience research. The computational tool developed in this study is based on the convolutional neural networks and the You Only Look Once (YOLO) algorithm for detecting and tracking mice in videos recorded during behavioral neuroscience experiments. The task of mice detection consists of determining the location in the image where the animals are present, for each frame acquired. In this work, we propose mice tracking using the YOLO algorithm, running on an NVIDIA GeForce GTX 1060 GPU. We analyzed a set of data composed of 13622 images, made up of behavioral videos of three important researches in this area. The training set used 50% of the images, 25% for validation and 25% for the tests. The results show that the mean Average Precision (mAP) reached by the developed system was 90.79% and 90.75% for the Full and Tiny versions of YOLO, respectively. It has also been found that the use of the Tiny version is a good alternative for experimental designs that require real-time response. Considering the high accuracy of the results, the developed work allows the experimentalists to perform mice tracking in a reliable and non-evasive way, avoiding common system errors that require delimitations of regions of interest (ROI) or even evasive luminous identifiers such as LED for tracking the animals.
The development of computational tools is essential for the development of new technologies, including experimental designs needed for behavioral neuroscience research. The computational tool developed in this study is based on the convolutional neural networks and the You Only Look Once (YOLO) algorithm for detecting and tracking mice in videos recorded during behavioral neuroscience experiments. The task of mice detection consists of determining the location in the image where the animals are present, for each frame acquired. In this work, we propose mice tracking using the YOLO algorithm, running on an NVIDIA GeForce GTX 1060 GPU. We analyzed a set of data composed of 13622 images, made up of behavioral videos of three important researches in this area. The training set used 50% of the images, 25% for validation and 25% for the tests. The results show that the mean Average Precision (mAP) reached by the developed system was 90.79% and 90.75% for the Full and Tiny versions of YOLO, respectively. It has also been found that the use of the Tiny version is a good alternative for experimental designs that require real-time response. Considering the high accuracy of the results, the developed work allows the experimentalists to perform mice tracking in a reliable and non-evasive way, avoiding common system errors that require delimitations of regions of interest (ROI) or even evasive luminous identifiers such as LED for tracking the animals.Machine learning approach for automated defense against network intrusionshttps://peerj.com/preprints/277772019-06-032019-06-03Farhaan Noor HamdaniFarheen Siddiqui
With the advent of the internet, there is a major concern regarding the growing number of attacks, where the attacker can target any computing or network resource remotely Also, the exponential shift towards the use of smart-end technology devices, results in various security related concerns, which include detection of anomalous data traffic on the internet. Unravelling legitimate traffic from malignant traffic is a complex task itself. Many attacks affect system resources thereby degenerating their computing performance. In this paper we propose a framework of supervised model implemented using machine learning algorithms which can enhance or aid the existing intrusion detection systems, for detection of variety of attacks. Here KDD (knowledge data and discovery) dataset is used as a benchmark. In accordance with detective abilities, we also analyze their performance, accuracy, alerts-logs and compute their overall detection rate.
These machine learning algorithms are validated and tested in terms of accuracy, precision, true-false positives and negatives. Experimental results show that these methods are effective, generating low false positives and can be operative in building a defense line against network intrusions. Further, we compare these algorithms in terms of various functional parameters
With the advent of the internet, there is a major concern regarding the growing number of attacks, where the attacker can target any computing or network resource remotely Also, the exponential shift towards the use of smart-end technology devices, results in various security related concerns, which include detection of anomalous data traffic on the internet. Unravelling legitimate traffic from malignant traffic is a complex task itself. Many attacks affect system resources thereby degenerating their computing performance. In this paper we propose a framework of supervised model implemented using machine learning algorithms which can enhance or aid the existing intrusion detection systems, for detection of variety of attacks. Here KDD (knowledge data and discovery) dataset is used as a benchmark. In accordance with detective abilities, we also analyze their performance, accuracy, alerts-logs and compute their overall detection rate.These machine learning algorithms are validated and tested in terms of accuracy, precision, true-false positives and negatives. Experimental results show that these methods are effective, generating low false positives and can be operative in building a defense line against network intrusions. Further, we compare these algorithms in terms of various functional parametersMachine learning of symbolic compositional rules with genetic programming: Dissonance treatment in Palestrinahttps://peerj.com/preprints/277312019-05-152019-05-15Torsten AndersBenjamin Inden
We describe a method to automatically extract symbolic compositional rules from music corpora that can be combined with each other and manually programmed rules for algorithmic composition, and some preliminary results of applying that method. As machine learning technique we chose genetic programming, because it is capable of learning formula consisting of both logic and numeric relations. Genetic programming was never used for this purpose to our knowledge. We therefore investigate a well understood case in this pilot study: the dissonance treatment in Palestrina’s music. We label dissonances with a custom algorithm, automatically cluster melodic fragments with labelled dissonances into different dissonance categories (passing tone, suspension etc.) with the DBSCAN algorithm, and then learn rules describing the dissonance treatment of each category with genetic programming. As positive examples we use dissonances from a given category. As negative examples we us all other dissonances; melodic fragments without dissonances; purely random melodic fragments; and slight random transformations of positive examples. Learnt rules circumstantiate melodic features of the dissonance categories very well, though some resulting best rules allow for minor deviations compared with positive examples (e.g., allowing the dissonance category suspension to occur also on shorter notes).
We describe a method to automatically extract symbolic compositional rules from music corpora that can be combined with each other and manually programmed rules for algorithmic composition, and some preliminary results of applying that method. As machine learning technique we chose genetic programming, because it is capable of learning formula consisting of both logic and numeric relations. Genetic programming was never used for this purpose to our knowledge. We therefore investigate a well understood case in this pilot study: the dissonance treatment in Palestrina’s music. We label dissonances with a custom algorithm, automatically cluster melodic fragments with labelled dissonances into different dissonance categories (passing tone, suspension etc.) with the DBSCAN algorithm, and then learn rules describing the dissonance treatment of each category with genetic programming. As positive examples we use dissonances from a given category. As negative examples we us all other dissonances; melodic fragments without dissonances; purely random melodic fragments; and slight random transformations of positive examples. Learnt rules circumstantiate melodic features of the dissonance categories very well, though some resulting best rules allow for minor deviations compared with positive examples (e.g., allowing the dissonance category suspension to occur also on shorter notes).Automated language essay scoring systems: A literature reviewhttps://peerj.com/preprints/277152019-05-092019-05-09Mohamed Abdellatif HusseinHesham Ahmed HassanMohamed Nassef
Background. Writing composition is a significant factor for measuring test-takers’ ability in any language exam. However, the assessment (scoring) of these writing compositions or essays is a very challenging process in terms of reliability and time. The need for objective and quick scores has raised the need for a computer system that can automatically grade essay questions targeting specific prompt. Automated Essay Scoring (AES) systems are used to overcome the challenges of scoring writing tasks by using Natural Language Processing and Machine Learning techniques. The purpose of this paper is to review the literature for the AES systems used for grading the essay questions.
Methodology. We have reviewed the existing literature using Google Scholar, EBSCO and ERIC to search the terms “AES”, “Automated Essay Scoring”, “Automated Essay Grading”, or “Automatic Essay”, and two categories have been identified: handcrafted features and automatic featuring AES systems. The systems of the first category are closely bonded to the quality of the designed features. On the other hand, the systems of the other category are based on the automatic learning of the features and relations between an essay and its score without any handcrafted features. We reviewed the systems of the two categories in terms of system primary focus, technique(s) used in the system, training data (y/n), instructional application (feedback system), and the correlation between e-scores and human scores. The paper is composed of three main sections. Firstly, we present a structured literature review of the available Handcrafted Features AES systems. Secondly, we present a structured literature review of the available Automatic Featuring AES systems. Finally, we draw a set of discussions and conclusions.
Results. AES models have been found to utilize a broad range of manually-tuned shallow and deep linguistic features. AES systems have many strengths in reducing labour-intensive marking activities, ensuring a consistent application of marking criteria, and facilitating equity in scoring. Although many techniques have been implemented to improve the AES systems, three primary challenges have been concluded: they lack the sense of the rater as a person, they can be tricked into assigning a lower or higher score to an essay than it deserved or not, and they cannot assess the creativity of the ideas and propositions and evaluating their practicality. Many techniques have been used to address the first two challenges only.
Background. Writing composition is a significant factor for measuring test-takers’ ability in any language exam. However, the assessment (scoring) of these writing compositions or essays is a very challenging process in terms of reliability and time. The need for objective and quick scores has raised the need for a computer system that can automatically grade essay questions targeting specific prompt. Automated Essay Scoring (AES) systems are used to overcome the challenges of scoring writing tasks by using Natural Language Processing and Machine Learning techniques. The purpose of this paper is to review the literature for the AES systems used for grading the essay questions.Methodology. We have reviewed the existing literature using Google Scholar, EBSCO and ERIC to search the terms “AES”, “Automated Essay Scoring”, “Automated Essay Grading”, or “Automatic Essay”, and two categories have been identified: handcrafted features and automatic featuring AES systems. The systems of the first category are closely bonded to the quality of the designed features. On the other hand, the systems of the other category are based on the automatic learning of the features and relations between an essay and its score without any handcrafted features. We reviewed the systems of the two categories in terms of system primary focus, technique(s) used in the system, training data (y/n), instructional application (feedback system), and the correlation between e-scores and human scores. The paper is composed of three main sections. Firstly, we present a structured literature review of the available Handcrafted Features AES systems. Secondly, we present a structured literature review of the available Automatic Featuring AES systems. Finally, we draw a set of discussions and conclusions.Results. AES models have been found to utilize a broad range of manually-tuned shallow and deep linguistic features. AES systems have many strengths in reducing labour-intensive marking activities, ensuring a consistent application of marking criteria, and facilitating equity in scoring. Although many techniques have been implemented to improve the AES systems, three primary challenges have been concluded: they lack the sense of the rater as a person, they can be tricked into assigning a lower or higher score to an essay than it deserved or not, and they cannot assess the creativity of the ideas and propositions and evaluating their practicality. Many techniques have been used to address the first two challenges only.Preliminary experiments with the Andean Condor Algorithm to solve problems of Continuous Domainshttps://peerj.com/preprints/276782019-04-242019-04-24Boris L Almonacid
In this article a preliminary experiment is carried out in which a set of elements and procedures are described to be able to solve problems of continuous domains integrated in the Andean Condor Algorithm. The Andean Condor Algorithm is a metaheuristic algorithm of swarm intelligence inspired by the movement pattern of the Andean condor when searching for its food. An experiment focused on solving the problem of the function 1st De Jong's \(f(x_1 \cdots x_n) = \sum_{i=1}^n x_i^2,~ -100 \leq x_i \leq 100\). According to the results obtained, solutions have been obtained close to the overall optimum value of the problem.
In this article a preliminary experiment is carried out in which a set of elements and procedures are described to be able to solve problems of continuous domains integrated in the Andean Condor Algorithm. The Andean Condor Algorithm is a metaheuristic algorithm of swarm intelligence inspired by the movement pattern of the Andean condor when searching for its food. An experiment focused on solving the problem of the function 1st De Jong's \(f(x_1 \cdots x_n) = \sum_{i=1}^n x_i^2,~ -100 \leq x_i \leq 100\). According to the results obtained, solutions have been obtained close to the overall optimum value of the problem.Improving rule based classification using harmony searchhttps://peerj.com/preprints/276342019-04-042019-04-04Hesam HasanpourRamak Ghavamizadeh MeibodiKeivan Navi
Classification and associative rule mining are two substantial areas in data mining. Some scientists attempt to integrate these two field called rule-based classifiers. Rule-based classifiers can play a very important role in applications such as fraud detection, medical diagnosis, and etc. Numerous previous studies have shown that this type of classifiers achieves high classification accuracy than traditional classification algorithms. However, they still suffer from a fundamental limitation. Many rule-based classifiers used various greedy techniques to prune the redundant rules that lead to missing some important rules. Another challenge that must be considered is related to the enormous set of mined rules that result in high processing overhead. The result of these approaches is that the final selected rules may not be the global best rules. These algorithms are not successful at exploiting search space effectively in order to select the best subset of candidate rules. We merged the Apriori algorithm, harmony search, and classification based association rules (CBA) algorithm for building a rule-based classifier. We applied a modified version of the Apriori algorithm with multiple minimum support for extracting useful rules for each class in the dataset. Instead of using a large number of candidate rules, binary harmony search was utilized for selecting the best subset of rules that appropriate for building a classification model. We applied the proposed method on seventeen benchmark dataset and compared its result with traditional association rule classification algorithms. The statistical results show that our proposed method outperformed other rule-based approaches.
Classification and associative rule mining are two substantial areas in data mining. Some scientists attempt to integrate these two field called rule-based classifiers. Rule-based classifiers can play a very important role in applications such as fraud detection, medical diagnosis, and etc. Numerous previous studies have shown that this type of classifiers achieves high classification accuracy than traditional classification algorithms. However, they still suffer from a fundamental limitation. Many rule-based classifiers used various greedy techniques to prune the redundant rules that lead to missing some important rules. Another challenge that must be considered is related to the enormous set of mined rules that result in high processing overhead. The result of these approaches is that the final selected rules may not be the global best rules. These algorithms are not successful at exploiting search space effectively in order to select the best subset of candidate rules. We merged the Apriori algorithm, harmony search, and classification based association rules (CBA) algorithm for building a rule-based classifier. We applied a modified version of the Apriori algorithm with multiple minimum support for extracting useful rules for each class in the dataset. Instead of using a large number of candidate rules, binary harmony search was utilized for selecting the best subset of rules that appropriate for building a classification model. We applied the proposed method on seventeen benchmark dataset and compared its result with traditional association rule classification algorithms. The statistical results show that our proposed method outperformed other rule-based approaches.Online transfer learning and organic computing for deep space research and astronomyhttps://peerj.com/preprints/275812019-03-122019-03-12Sadanandan Natarajan
Deep space exploration is the pillars within the field of outer space analysis and physical science. The amount of knowledge from numerous space vehicle and satellites orbiting the world of study are increasing day by day. This information collected from numerous experiences of the advanced space missions is huge. These information helps us to enhance current space knowledge and the experiences can be converted and transformed into segregated knowledge which helps us to explore and understand the realms of the deep space.. Online Transfer Learning (OTL) is a machine learning concept in which the knowledge gets transferred between the source domain and target domain in real time, in order to help train a classifier of the target domain. Online transfer learning can be an efficient method for transferring experiences and data gained from the space analysis data to a new learning task and can also routinely update the knowledge as the task evolves.
Deep space exploration is the pillars within the field of outer space analysis and physical science. The amount of knowledge from numerous space vehicle and satellites orbiting the world of study are increasing day by day. This information collected from numerous experiences of the advanced space missions is huge. These information helps us to enhance current space knowledge and the experiences can be converted and transformed into segregated knowledge which helps us to explore and understand the realms of the deep space.. Online Transfer Learning (OTL) is a machine learning concept in which the knowledge gets transferred between the source domain and target domain in real time, in order to help train a classifier of the target domain. Online transfer learning can be an efficient method for transferring experiences and data gained from the space analysis data to a new learning task and can also routinely update the knowledge as the task evolves.Pattern recognition techniques for the identification of Activities of Daily Living using mobile device accelerometerhttps://peerj.com/preprints/272252019-02-122019-02-12Ivan Miguel PiresNuno M. GarciaNuno PomboFrancisco Flórez-RevueltaSusanna SpinsanteMaria Canavarro TeixeiraEftim Zdravevski
This paper focuses on the recognition of Activities of Daily Living (ADL) applying pattern recognition techniques to the data acquired by the accelerometer available in the mobile devices. The recognition of ADL is composed by several stages, including data acquisition, data processing, and artificial intelligence methods. The artificial intelligence methods used are related to pattern recognition, and this study focuses on the use of Artificial Neural Networks (ANN). The data processing includes data cleaning, and the feature extraction techniques to define the inputs for the ANN. Due to the low processing power and memory of the mobile devices, they should be mainly used to acquire the data, applying an ANN previously trained for the identification of the ADL. The main purpose of this paper is to present a new method based on ANN for the identification of a defined set of ADL with a reliable accuracy. This paper also presents a comparison of different types of ANN in order to choose the type for the implementation of the final model. Results of this research probes that the best accuracies are achieved with Deep Neural Networks (DNN) with an accuracy higher than 80%. The results obtained are similar with other studies, but we compared tree types of ANN in order to discover the best method in order to obtain these results with less memory, verifying that, after the generation of the model, the DNN method, when compared with others, is also the fastest to obtain the results with better accuracy.
This paper focuses on the recognition of Activities of Daily Living (ADL) applying pattern recognition techniques to the data acquired by the accelerometer available in the mobile devices. The recognition of ADL is composed by several stages, including data acquisition, data processing, and artificial intelligence methods. The artificial intelligence methods used are related to pattern recognition, and this study focuses on the use of Artificial Neural Networks (ANN). The data processing includes data cleaning, and the feature extraction techniques to define the inputs for the ANN. Due to the low processing power and memory of the mobile devices, they should be mainly used to acquire the data, applying an ANN previously trained for the identification of the ADL. The main purpose of this paper is to present a new method based on ANN for the identification of a defined set of ADL with a reliable accuracy. This paper also presents a comparison of different types of ANN in order to choose the type for the implementation of the final model. Results of this research probes that the best accuracies are achieved with Deep Neural Networks (DNN) with an accuracy higher than 80%. The results obtained are similar with other studies, but we compared tree types of ANN in order to discover the best method in order to obtain these results with less memory, verifying that, after the generation of the model, the DNN method, when compared with others, is also the fastest to obtain the results with better accuracy.