基于残差注意力的新槽值抽取研究

NEW SLOT VALUE EXTRACTION BASED ON RESIDUAL ATTENTION

  • 摘要: 针对任务型对话中用户输入存在新槽值难以提取的问题,提出一种基于双向长短时记忆神经网络的特征融合模型。通过负样本添加技术缓解新槽值识别问题;引入注意力机制来提取用户输入中词与词之间的权重特征;在多层堆叠长短时记忆神经网络之间用残差连接以缓解特征提取过程中的信息丢失,JP+1将得到的融合特征输入到条件随机场筛选出合理的标注序列。在DSTC3数据集上进行相关测试,实验结果表明,该模型在新槽值、已知槽值和总数据集上的识别准确率分别为70.73%、91.56%和87.11%,性能相比baseline方法有显著提高。

     

    Abstract: In order to solve the problem that new slot values are difficult to be extracted from user input in task-oriented dialogue, a feature fusion model based on bidirectional long and short-term memory network is proposed. The negative sample addition technique was used to alleviate the problem of new slot value recognition. The attention mechanism was introduced to extract the weight features between words in user input. The residuals were used to link the multi-layer stacked LSTM networks to alleviate the loss of information in the process of feature extraction. The fused features were input into the conditional random field to screen out the reasonable labeling sequence. In this paper, we tested the model on DSTC3 dataset. The experimental results show that the recognition rates of this model on new slot values, known slot values and total data sets are 70.73%, 91.56% and 87.11%, respectively. The performance of this model is significantly better than that of baseline method.

     

/

返回文章
返回