BERT reaches 81.3% accuracy in outage detection test

Olukunle Owolabi’s arXiv preprint, submitted May 28, 2023, reported that BERT reached 81.3% accuracy in a power-outage detection task after fine-tuning on 10% of the available training data.

The comparison matters for utilities and emergency managers because outage reports can arrive before formal operational data. In the May 28, 2023 arXiv preprint, Owolabi said GPT reached 74.5% accuracy under the same 10% fine-tuning setup.

The paper tested language models on a curated, balanced dataset of social-media tweets related to power outages. Zero-shot learning means applying a model without task-specific examples; few-shot learning means adapting it with a small amount of labeled task data.

The abstract said classical machine-learning models outperformed zero-shot language models in the experiments. Fine-tuning changed the result: BERT’s 81.3% accuracy was reported as a 15.3% improvement, and GPT’s 74.5% accuracy was reported as an 8.5% improvement.

The paper frames the work as a way to detect and localize outages when labeled data are scarce. The abstract said the results have “practical implications for analyzing and localizing outages in scenarios with limited data availability.”

The infrastructure context is clear, even if the paper is a limited research result. Atlas360 reported on Oct. 19 that Ukrainian officials said Russian missile attacks had left 500,000 people without power, and reported in August that Tropical Storm Lala left more than 60,000 homes without power in Hawaii.

The findings should not be read as evidence of a deployed outage-monitoring system.

The result also leaves a practical trade-off unresolved. Social-media signals can be fast, but the abstract does not say how the models handle sparse posting, misleading reports or areas where residents do not use the same platforms during an outage.

The paper’s contribution is a benchmark result showing that limited fine-tuning improved language-model performance on tweet-based outage detection.

Source: arXiv preprint, arXiv cs.AI, May 28, 2023.