QG Survey

QG Survey


This post mainly consists of papers on Question Generation (QG) from ACL’20 and controllable diversity.

Learn from Your Neighbor: Learning Multimodal Mapping from Sparse Annotations

Summary & Intuitions

  • place not penalization but brief on plausible predictions
  • inspired by semi-supervised learning, annotated neighbors are important signals for diversity
  • neighbor definition and neighbor penalization


  • neighbor definition: semantic space and distance metrics (similarity)
  • neighbor penalization:
    • label-missing multi-label (or multi-way) classification: similarity as loss weight
    • sequence generation from image or text input:
      • overall weight: similarity from double inputs
      • current token attends neighbor input for: (address the issue of unrelated visual objects and semantic tokens)
        • token-wise weight
        • modulated sequence generation of neighbors (language attention)
        • choose image region or feature (visual attention)
      • note: non-trivial derivatives of weights is necessary