FoodGPT: A Large Language Model in Food Testing Domain with Incremental Pre-training and Knowledge Graph Prompt

by   Zhixiao Qi, et al.

Currently, the construction of large language models in specific domains is done by fine-tuning on a base model. Some models also incorporate knowledge bases without the need for pre-training. This is because the base model already contains domain-specific knowledge during the pre-training process. We build a large language model for food testing. Unlike the above approach, a significant amount of data in this domain exists in Scanning format for domain standard documents. In addition, there is a large amount of untrained structured knowledge. Therefore, we introduce an incremental pre-training step to inject this knowledge into a large language model. In this paper, we propose a method for handling structured knowledge and scanned documents in incremental pre-training. To overcome the problem of machine hallucination, we constructe a knowledge graph to serve as an external knowledge base for supporting retrieval in the large language model. It is worth mentioning that this paper is a technical report of our pre-release version, and we will report our specific experimental data in future versions.


page 1

page 2

page 3

page 4


TravelBERT: Pre-training Language Model Incorporating Domain-specific Heterogeneous Knowledge into A Unified Representation

Existing technologies expand BERT from different perspectives, e.g. desi...

Task-specific Pre-training and Prompt Decomposition for Knowledge Graph Population with Language Models

We present a system for knowledge graph population with Language Models,...

Semi-Automated Construction of Food Composition Knowledge Base

A food composition knowledge base, which stores the essential phyto-, mi...

Constructing a Knowledge Graph from Unstructured Documents without External Alignment

Knowledge graphs (KGs) are relevant to many NLP tasks, but building a re...

2x Faster Language Model Pre-training via Masked Structural Growth

Acceleration of large language model pre-training is a critical issue in...

Contextual Mood Analysis with Knowledge Graph Representation for Hindi Song Lyrics in Devanagari Script

Lyrics play a significant role in conveying the song's mood and are info...

Unifying Structure Reasoning and Language Model Pre-training for Complex Reasoning

Recent knowledge enhanced pre-trained language models have shown remarka...

Please sign up or login with your details

Forgot password? Click here to reset