Performance Analysis of Machine Learning Models for the Detection of Cyber Threats Against Satellite Networks

Abstract

The proliferation of satellite networks for various critical applications in the space sector has heightened the need for robust cybersecurity measures to safeguard these systems from malicious intrusions. An intrusion detection system serves as the backbone for providing high-level network security. Different network attacks have been discovered and are gradually becoming more sophisticated and complicated. Despite the availability of many existing intrusion detection systems, intuitive cybersecurity systems are needed due to alarmingly increasing intrusion attacks. Furthermore, with new intrusion attacks, the efficacy of existing systems is depleted unless they evolve. This study conducts experiments that compare three types of supervised machine learning algorithms, including Decision Tree (CART), SVM (Black-box), and KNN (Lazy learner). Thus, these different algorithms were compared using various evaluation metrics, which are accuracy, recall, false alarm rate, and precision, and manual feature selections were done to select important features from the dataset that increase relevance and reduce complexity along with the Training time complexity on three intrusion datasets (STIN, UNSW-NB15, and CIC IDS2017(Wednesday)). CART DT achieves an accuracy of 93.42% with 31 features of the STIN dataset in 63.63 seconds, an accuracy of 93.13% with 8 features in 6.22 seconds, 76.63% with 42 features of the UNSW-NB15 dataset in 48.5 seconds, 76.63% with 6 features in 3.71 seconds, 99.87% with 68 features of the CIC-IDS 2017 Wednesday dataset in 59.29 seconds, 95.80% with 16 features in 4.96 seconds. SVM achieves an accuracy of 87.41% with 31 features of the STIN dataset in 559.51 seconds, an accuracy of 87.04% with 8 features in 286.78 seconds, 81.51% with 194 features of the UNSW-NB15 dataset in 421.06 seconds, 78.90% with 6 features in 176.05 seconds, 98.48% with 68 features of the CIC-IDS 2017 Wednesday dataset in 505.13 seconds, 96.92% with 16 features in 209.45 seconds. KNN achieves an accuracy of 86.28% with 194 features of the UNSW-NB15 dataset in 236.09 seconds. The results of this experiment give valuable insight for machine learning researchers into building a time-efficient and effective IDS using supervised machine learning for the Space sector. Although the secondary datasets used in this study provided good results, the use of primary datasets is suggested to enhance and improve the accuracy, integrity, and real-timeliness of the threat intelligence and resilience of the satellite networks in the space industry.