Issues

• infbench_choice marks empty or wrong answers correct when the gold letter is "A", because normalize_answer turns "A" into ""

  princeton-nlp/HELMET#48 · Filter by author damienen opened on Nov 2, 2026

• eval_gpt4_longqa.py: gpt-4-score averages only judged rows without recording how many were dropped, and the same block has three crash paths

  princeton-nlp/HELMET#46 · Filter by author shaurya416 opened on Sep 23, 2026

• Optional EvalPort export for HELMET's per-item results / averaged_metrics

  princeton-nlp/HELMET#45 · Filter by author adhabnr-ux opened on Sep 22, 2026
  3

• Understanding file name and content

  princeton-nlp/HELMET#31 · Filter by author framolfese opened on Aug 20, 2025
  1

• How to understand the mearning of hotpotqa-dev-multikilt_1000_k1000_dep3.jsonl

  princeton-nlp/HELMET#27 · Filter by author lemyx opened on Jul 16, 2025
  3

• load_qa takes too much time

  princeton-nlp/HELMET#21 · Filter by author yjsoh opened on Mar 31, 2025
  1

• Reproducing results on Llama-3.1-8B-Inst

  princeton-nlp/HELMET#8 · Filter by author chtmp223 opened on Oct 22, 2024
  3

• ALCE citation evaluation

  princeton-nlp/HELMET#7 · Filter by author carriex opened on Oct 22, 2024
  3

• Discrepancy in gpt4o-mini Results on MSMarco Compared to Reported Results

  princeton-nlp/HELMET#6 · Filter by author 8188zq opened on Oct 22, 2024
  9