Describe the tutorial's historical SciBERT comparison

This commit is contained in:
Andras Schmelczer 2026-10-27 21:59:13 +00:00
commit 6bbfeadf9f

View file

@ -234,7 +234,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"Great work, we can be rightfully satisfied with our model. Seeing the results, we achieved an F1-score of 0.69 which is about **5% better than SciBERT's** 0.6571!\n",
"The recorded run achieves a test macro F1-score of about **0.69** on the MAG sentence-classification dataset, above the historical SciBERT baseline of **0.6571** associated with the [2019 SciBERT work](https://arxiv.org/abs/1903.10676). This compares a linear SVM with one historical baseline; it does not establish current state-of-the-art performance. A broader claim would require a current benchmark with comparable data splits, preprocessing, and metrics.\n",
"\n",
"You might wonder that *\"this is great, but besides some utility functions (`clean`, `simple_parallel_map`, ...) what more value does GreatAI add?\"*. This would be a valid argument because the scope of GreatAI actually only starts here.\n",
"\n",