Open Internet by MindsNet

MobileBERT's Poor Performance on Tabular Fault Detection Data

MobileBERT consistently scored 0 F1 across three fault-detection datasets when used for tabular data, unlike DistilBERT and TinyBERT which achieved strong results. The issue seems specific to MobileBERT's architecture when handling tabular features converted into text tokens. The cause may be related to the model's bottleneck architecture discarding fine-grained numerical information.

Computing & Technology, Computer Science, Machine Learning