QG Survey

Overview

This post mainly consists of papers on Question Generation (QG) from ACL’20 and controllable diversity.

Learn from Your Neighbor: Learning Multimodal Mapping from Sparse Annotations

Summary & Intuitions

• place not penalization but brief on plausible predictions
• inspired by semi-supervised learning, annotated neighbors are important signals for diversity
• neighbor definition and neighbor penalization

Contributions

• neighbor definition: semantic space and distance metrics (similarity)
• neighbor penalization:
  • label-missing multi-label (or multi-way) classification: similarity as loss weight
  • sequence generation from image or text input:
    • overall weight: similarity from double inputs
    • current token attends neighbor input for: (address the issue of unrelated visual objects and semantic tokens)
      • token-wise weight
      • modulated sequence generation of neighbors (language attention)
      • choose image region or feature (visual attention)
    • note: non-trivial derivatives of weights is necessary