QG Survey
Overview
This post mainly consists of papers on Question Generation (QG) from ACL’20 and controllable diversity.
Learn from Your Neighbor: Learning Multimodal Mapping from Sparse Annotations
Summary & Intuitions
- place not penalization but brief on plausible predictions
- inspired by semi-supervised learning, annotated neighbors are important signals for diversity
- neighbor definition and neighbor penalization
Contributions
- neighbor definition: semantic space and distance metrics (similarity)
- neighbor penalization:
- label-missing multi-label (or multi-way) classification: similarity as loss weight
- sequence generation from image or text input:
- overall weight: similarity from double inputs
- current token attends neighbor input for: (address the issue of unrelated visual objects and semantic tokens)
- token-wise weight
- modulated sequence generation of neighbors (language attention)
- choose image region or feature (visual attention)
- note: non-trivial derivatives of weights is necessary
PREVIOUSLM Survey
NEXTGNN Survey