基于Deberta和语篇图神经网络的机器阅读理解

MACHINE READING COMPREHENSION BASED ON DEBERTA AND DISCOURSE GRAPH NEURAL NETWORKS

  • 摘要: 机器阅读理解多项选择式任务在自然语言处理领域广受关注,但是现有的预训练模型在逻辑推理型多项选择式任务中效果还有待提升。基于Deberta和语篇图神经网络提出一种改进的Deberta-DGNN(Deberta-DiscourseGraphNeuralNetwork)模型。使用Deberta模型进行词向量的特征提取;通过构建图的形式来完成句子间隐藏关系的提取,将Deberta模型输出的序列特征进行切割作为语篇单元来构建逻辑图。为保持原文原意,在逻辑图中对语篇单元补入位置信息。对于逻辑图长距离依赖节点难以进行有效交互的问题,将节点引入多头自注意力机制中来缓解该问题。该模型在Reclor数据集上测试准确率达到68.40%,效果提升显著。

     

    Abstract: Machine reading comprehension multiple-choice tasks have attracted wide attention in the field of natural language processing, but the effectiveness of existing pre-trained models in logical reasoning multiple-choice tasks needs to be improved. An improved Deberta-Discourse graph neural network (Deberta-DGNN) model is proposed based on Deberta and Discourse Graph Neural Network. The Deberta model was used to extract the feature of the word vector. The hidden relationship between sentences was extracted by constructing a graph, and the sequence features output by the Deberta model were cut as discourse units to construct a logical graph. In order to keep the original meaning of the original text, position information was added to the discourse unit in the logic diagram. For the problem that the long-distance dependent nodes of the logic graph were difficult to interact effectively, the node was introduced into the multi-head self-attention mechanism to alleviate this problem. The proposed model has a test accuracy of 68.40% on the Reclor dataset, and the effect has been significantly improved.

     

/

返回文章
返回