TDA_2023_demo_tasks_4.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "provenance": [],
      "authorship_tag": "ABX9TyMCfmTzO1oDC413Kj8RftO9",
      "include_colab_link": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/TurkuNLP/textual-data-analysis-course/blob/main/TDA_2023_demo_tasks_4.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Tasks for the 4th set of exercises"
      ],
      "metadata": {
        "id": "kgn1ofUdtxb_"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Task 1\n",
        "\n",
        "This week, we looked into datasets for sequence labeling ([notebook](https://github.com/TurkuNLP/textual-data-analysis-course/blob/main/sequence_labeling_dataset_examples.ipynb)) and studied an example of named entity recognition ([notebook](https://github.com/TurkuNLP/textual-data-analysis-course/blob/main/sequence_labeling_example.ipynb)).Building on these resources and the documentation referenced below, do the following:\n",
        "\n",
        "1. Load a pipeline with a model that has previously been fine-tuned on the `conll2003` dataset and evaluate it on the `test` part of the `conll2003` corpus. For reference, see the [named entity recognition](https://huggingface.co/docs/transformers/v4.17.0/en/task_summary#named-entity-recognition) and [evaluator](https://huggingface.co/docs/evaluate/base_evaluator#using-the-evaluator) documentation. You can use [this template](https://github.com/TurkuNLP/textual-data-analysis-course/blob/main/tda_2023_exercises_4_1_template.ipynb) as a starting point.\n",
        "What `overall_f1` do you get as a result?\n",
        "\n",
        "2. Evaluate the same model on the `test` part of the `turku_ner_corpus` dataset. What `overall_f1` do you get as a result? What reasons can you find for this result?\n",
        "\n",
        "3. Train and evaluate a better NER tagger for `turku_ner_corpus`. You can use the [sequence labeling example notebook](https://github.com/TurkuNLP/textual-data-analysis-course/blob/main/sequence_labeling_example.ipynb) as a starting point. Try at least one of the following options: a) train a multilingual model on the `conll2003` data, b) train a multilingual model on the `turku_ner_corpus` data, c) train a monolingual model on the `turku_ner_corpus_data`.\n"
      ],
      "metadata": {
        "id": "hH4X-eenwUxe"
      }
    }
  ]
}