# LM Survey

## Overview

Let’s go through the state-of-the-art models at the time and have a comprehensive understanding of transfer learning in NLP.

# QG Survey

## Overview

This post mainly consists of papers on Question Generation (QG) from ACL’20 and controllable diversity.

## Learn from Your Neighbor: Learning Multimodal Mapping from Sparse Annotations

### Summary & Intuitions

• place not penalization but brief on plausible predictions
• inspired by semi-supervised learning, annotated neighbors are important signals for diversity
• neighbor definition and neighbor penalization

### Contributions

• neighbor definition: semantic space and distance metrics (similarity)
• neighbor penalization:
• label-missing multi-label (or multi-way) classification: similarity as loss weight
• sequence generation from image or text input:
• overall weight: similarity from double inputs
• current token attends neighbor input for: (address the issue of unrelated visual objects and semantic tokens)
• token-wise weight
• modulated sequence generation of neighbors (language attention)
• choose image region or feature (visual attention)
• note: non-trivial derivatives of weights is necessary

[TOC]

# GNN Survey

## Overview

In my opinion, graph neural networks is worth exploring within research field, not engineering field, especially large scale application. Anyway, this survey is a naive summarization of my recent reading. Writing in English is just a practice of my poor writing ability.

Without whistles and bells, there are some equivalent concepts or notions in graph terminology. First of all, graph signal is general representation, which stands for collection of node embeddings or node signals whatever domain we talk about. Graph Fourier Transformation (GFT) is just discrete version Fourier Transformation (FT). Fourier Transformation (FT) converts temporal signals to frequency signal with the help of operator $e^{-iwt}$ (sine, cosine base function), and vice versa. In a similar way, GFT converts spatial domain to spectral domain with the help of $\phi^T$ (eigen base vector).

Note spatial domain is also known as vertex domain , graph domain or data space domain. In the contrast, Spectral domain is also denoted as feature space domain.

# CPP-Exception

## some moral or ethics

// Example 2(b): Very Buggy Class
//
class X : Y {
T* t_;
Z* z_;
public:
X()
try
: Y(1)
, t_( new T( static_cast<Y*>(this) )
, z_( new Z( static_cast<Y*>(this), t_ ) )
{
/*...*/
}
catch(...)
// Y::Y or T::T or Z::Z or X::X's body has thrown
{
// Q: should I delete t_ or z_? (note: not legal C++)
}
};

Therefore the status quo can be summarized as follows:

Moral #1: Constructor function-try-block handlers have only one purpose – to translate an exception. (And maybe to do logging or some other side effects.) They are not useful for any other purpose.

Moral #2: Since destructors should never emit an exception, destructor function-try-blocks have no practical use at all.[6] There should never be anything for them to detect, and even if there were something to detect because of evil code, the handler is not very useful for doing anything about it because it can not suppress the exception.

Moral #3: Always perform unmanaged resource acquisition in the constructor body, never in initializer lists. In other words, either use “resource acquisition is initialization” (thereby avoiding unmanaged resources entirely) or else perform the resource acquisition in the constructor body.

For example, building on Example 2(b), say T was char and t_ was a plain old char* that was new[]’d in the initializer-list; then in the handler there would be no way to delete[] it. The fix would be to instead either wrap the dynamically allocated memory resource (e.g., change char* to string) or new[] it in the constructor body where it can be safely cleaned up using a local try-block or otherwise.

Moral #4: Always clean up unmanaged resource acquisition in local try-block handlers within the constructor or destructor body, never in constructor or destructor function-try-block handlers.

Moral #5: If a constructor has an exception specification, that exception specification must allow for the union of all possible exceptions that could be thrown by base and member subobjects. As Holmes might add, “It really must, you know.” (Indeed, this is the way that the implicitly generated constructors are declared; see GotW #69.)

Moral #6: If a constructor of a member object can throw but you can get along without said member, hold it by pointer and use the pointer’s nullness to remember whether you’ve got one or not, as usual. Use the Pimpl idiom to group such “optional” members so you only have to allocate once.

And finally, one last moral that overlaps with the above but is worth restating in its own right:

Moral #7: Prefer using “resource acquisition is initialization” to manage resources. Really, really, really. It will save you more headaches than you can probably imagine.

• ARIES algorithm

aka $[‘eriz]$ 白羊宫

• A Transaction Recovery Method Supporting Fine Granularity Locking and Partial Rollback Using Write Ahead Logging

# EM-GMM

• 无监督学习
• 前提 数据分布假设
• 目标 数据分布拟合, 分布规律或分布模式的拟合

$\min_{\omega, b}\frac{1}{2}||w||^2 \\ s.t. (y_i(\omega^Tx_i+b) - 1) \ge 0\\ \alpha_i \ge 0 \\ 拉格朗日最值充分条件 => 对偶问题 \\ L() = \frac{1}{2}||w||^2 + \sum_{i = 1}^{m}\alpha_i(y_i(\omega^Tx_i+b) - 1) \\ \omega = \sum_{i=1}^{m}\alpha_iy_ix_i \\ 0 = \sum_{i = 1}^{m}\alpha_iy_i \\ 整理得 \\ \max_{\alpha} \sum_{i = 1}^{m}\alpha_i - \frac{1}{2}\sum_{i=1}^{m}\sum_{j=1}^{m}\alpha_i\alpha_jy_iy_jx_i^Tx_j \\ s.t. \alpha_i \ge 0 \\ 0 = \sum_{i = 1}^{m}\alpha_iy_i \\$