NIST TAC Knowledge Base Population (KBP2020) Entity Discovery and Linking Track

  • KBP Entity Discovery and Linking Scorer written by Ben Hachey and Joel Nothman
      • 1. Clustering Performance:
        For all mentions, we will ignore their KB IDs, use three measures to compute clustering performance:
        (1) CEAF: Official measure because it's technically more sound than B-cubed. We will use CEAF-m (mention level) instead of CEAF-e (entity level) because CEAF-m is more sensitive to cluster size.
        (2) B-cubed: in order to compare with results from previous years

        2. Linking Performance:
        We will use a standard Wikification F-measure to evaluate the performance of linking Non-NIL mentions to KB. A mention's label is correct only if its offsets, type and KB id are all correct.