Can AI detect a silent Internet outage before it occurs?

Istanbul, Türkiye – The Internet can encounter a strange type of failure where a router seems to function normally but silently drops some data packets. Network engineers refer to these issues as ''black holes.'' In these cases, data enters the network yet some of it vanishes without the router informing the network or the sender. Often, these problems go unnoticed until users start experiencing connection issues or data loss. 

Researchers from Istanbul Technical University (ITU) have created an AI-based method to identify warning signs linked to silent failures in networks. This approach aims to shift from merely detecting packet loss to recognizing early indicators of potential failures. Published in IEEE Transactions on Network and Service Management, the study integrates various machine-learning techniques to monitor how backbone network traffic evolves over time. Instead of just spotting obvious issues, it detects subtle warning patterns that can be hard to distinguish from normal network activity. If further refined for operational use, this forecasting method could provide network operators with more lead time to investigate problems, reroute traffic, or take preventive measures before users experience connectivity loss. 

When normal-looking traffic signals a problem

Black holes are difficult to predict because they often lack clear warning signs. A sudden and large spike in network traffic is usually easy to spot as abnormal. However, other issues are more subtle. For example, a traffic measurement might seem normal alone but appear suspicious if it occurs at an unusual time or deviates unexpectedly from nearby traffic. Additionally, individual measurements might seem normal, but when combined, they can form an unusual pattern. These are known as point, contextual, and collective anomalies. Past research on backbone black holes targeted specific abnormal behaviors, but the new approach aims to include all three types within a single predictive system. This system doesn’t just look for obvious problems but also evaluates if current activity aligns with recent traffic patterns. This helps detect warning signs that might otherwise go unnoticed in normal-looking network traffic.

Learning from failures that are hard to see

Predicting black holes is challenging because collecting confirmed examples is difficult. Black holes caused by hardware failures, misconfigurations, routing issues, or implementation errors often occur without clear notifications, leaving limited historical data. To address this, the researchers developed a method that learns from unlabeled network traffic. Instead of requiring engineers to label thousands of instances as 'normal” or “black hole,” the system detects unusual behaviors directly within the traffic. It works in stages: first, it identifies the most obvious abnormalities. Instead of removing these, it temporarily smooths them to help the AI understand typical traffic patterns over time. Then, it analyzes sequences to find subtle warning signs that only emerge in context. Technically, this combines clustering, time-series analysis, and deep learning in a three-stage forecasting model.

Testing the approach on network traffic

The researchers tested their method using both actual ISP backbone traffic and recognized public network datasets. Their real-world dataset included 17,280 observations taken every five minutes over 60 days. Since real black-hole incidents are inherently hard to detect and label, these data couldn't provide a definitive list of known black holes for validation. To establish quantitative benchmarks, they also used two well-known labeled network anomaly datasets, UNSW-NB15 and ToN_IoT, converting them into time-series format so their model's performance could be compared with other unsupervised anomaly detection methods. These public datasets feature different kinds of abnormal and malicious network activities, not actual black-hole incidents, which is important for interpreting the results. On the labeled datasets, their approach achieved a detection rate of up to 98% and an F1 score of around 90%, exceeding the performance of other unsupervised forecasting models in the comparison. The team also tested their method on the unlabeled real-world ISP data. The experiments showed that their sliding-window approach could identify black-hole patterns shortly before they happened, within the five-minute forecast window.

"The challenge with these failures is that nothing necessarily appears broken. A router can keep functioning while packets quietly vanish, so by the time it’s obvious there's a problem, users might already be affected. Our goal is to shift network management towards predicting failures rather than just reacting," said Kiymet Kaya, the study’s corresponding author.

From reacting to anticipating

The researchers view this work as a step toward developing network management systems that can identify emerging issues rather than just reporting failures after they happen. With adequate warning, network operators might investigate an affected router, reroute traffic, or take other preventive actions before a black hole causes a prolonged communication disruption. However, the study does not yet prove the system can prevent real-world Internet outages. Its effectiveness may vary depending on the network and training data, and modifications to network infrastructure might require retraining the model. Future research should examine the approach across a wider range of network types and structures, explore automated retraining as networks evolve, and incorporate explainable AI techniques to help operators understand why certain traffic patterns are flagged as suspicious. Ultimately, the goal is to shift from detecting packet loss to recognizing early warning signs of potential failure, improving how difficult-to-detect network issues are managed.