# Conversations Are Not Flat Modeling the Dynamic Information Flow across Dialogue Utterances

## Method

$$\mathbf{I}_{k}=\mathbf{C}_{k+1}-\mathbf{C}_{k}$$
DialoFlow首先对对话历史进行编码，并根据历史上下文$C_{1},…,C_{k}$预测未来的上下文$C^{‘}_{k+1}$。然后在回复生成阶段，模型预测 semantic influence $I^{‘}_{k}$，并结合$I^{‘}_{k}$和历史子句生成目标回复$u_{k}$。

$$\mathbf{C}_{k+1}^{\prime}=\operatorname{Flow}\left(\mathbf{C}_{1}, \mathbf{C}_{2}, \ldots, \mathbf{C}_{k}\right)$$

$$\mathbf{I}_{k}^{\prime}=\mathbf{C}_{k+1}^{\prime}-\mathbf{C}_{k}$$

• Context Flow Modeling: $$\mathcal{L}_{C F M}=\sum_{k=1}^{N}\left|\mathbf{C}_{k}-\mathbf{C}_{k}^{\prime}\right|_{2}^{2}$$
• Semantic Influence Modeling: \begin{aligned} \mathcal{L}_{S I M} &=-\sum_{k=1}^{N} \sum_{t=1}^{T} \log p\left(u_{k}^{t} \mid \mathbf{I}_{k}^{\prime}\right) \\ &=-\sum_{k=1}^{N} \sum_{t=1}^{T} \log f_{u_{k}^{t}} \end{aligned}
$$f=\operatorname{softmax}\left(W_{2} \mathbf{I}_{k}^{\prime}+b_{2}\right) \in \mathbb{R}^{|V|}$$
• \begin{aligned} \mathcal{L}_{R G M} &=-\sum_{k=1}^{N} \log p\left(u_{k} \mid \mathbf{I}_{k}^{\prime}, u_{<k}\right) \\ &=-\sum_{k=1}^{N} \sum_{t=1}^{T} \log p\left(u_{k}^{t} \mid \mathbf{I}_{k}^{\prime}, u_{<k}, u_{k}^{<t}\right) \end{aligned}

## Flow Score

\begin{aligned} s_{k} &=\cos \left(\left\langle\mathbf{I}_{k}^{\prime}, \mathbf{I}_{k}\right\rangle\right) \cdot \operatorname{length}\left(\mathbf{I}_{k}^{\prime}, \mathbf{I}_{k}\right) \\ &=\frac{\mathbf{I}_{k}^{\prime} \cdot \mathbf{I}_{k}}{\left|\mathbf{I}_{k}^{\prime}\right|\left|\mathbf{I}_{k}\right|} \cdot \frac{\min \left(\left|\mathbf{I}_{k}^{\prime}\right|,\left|\mathbf{I}_{k}\right|\right)}{\max \left(\left|\mathbf{I}_{k}^{\prime}\right|,\left|\mathbf{I}_{k}\right|\right)} \end{aligned}

$$\text { Flow score }=2^{-\frac{1}{M} \sum_{k}^{M} \log \left(\frac{s_{k}+1}{2}\right)}$$