Domain-tailored RAG framework improves industrial LLM question answering in new engineering study

A research team from multiple Chinese academic institutions has published a domain-adapted retrieval-augmented generation (RAG) system in Engineering, addressing persistent factual hallucination and insufficient domain knowledge integration issues that limit large language model (LLM) performance on industrial technical queries. Modern industrial operations rely heavily on knowledge-based inquiries across manufacturing, equipment maintenance and fault diagnosis scenarios, yet conventional LLMs often produce fluent but factually incorrect outputs when handling specialized industrial content, while earlier corpus-based and knowledge graph-based question answering (QA) systems carry limitations in semantic comprehension or large-scale data management. The proposed multi-module industrial RAG pipeline targets these pain points through four core technical contributions laid out in the peer-reviewed work.

The research first details the construction of a segmented industrial knowledge base assembled from five source categories: journal articles, dissertations, monographs, patents and production documents. Raw materials undergo text conversion, regular expression cleaning, deduplication, tokenization and chunking, with each text segment capped at 256 characters and set with a 32-character overlap to retain semantic continuity. The unified knowledge repository is split into five independent sub-knowledge bases corresponding to chemical engineering, petroleum, metallurgy, electricity and building materials sectors, laying a partitioned retrieval foundation to narrow search scope for industrial queries.

A BERT-based text classifier serves as the pipeline’s front-end filter, conducting dual classification tasks on incoming user questions. It first runs binary screening to exclude non-industrial irrelevant queries before assigning valid industrial questions to their matching sub-domain knowledge partitions. The classifier extracts semantic features from the BERT token embedding and feeds the 768-dimensional hidden representation into a two-layer fully connected network to output six category labels covering five industrial branches plus a non-relevance tag. Comparative tests against FastText, TextCNN, TextRNN and TextRCNN baselines show the BERT classifier delivers steady classification performance with low result variance across repeated training runs.

For knowledge retrieval, the study adopts a GTE–DPR dual-encoder dense retrieval architecture, replacing standard BERT encoders with the GTE-large-zh embedding model to generate 1024-dimensional text vectors. FAISS library powers L₂ distance-based vector similarity matching between query embeddings and precomputed knowledge chunk embeddings, followed by a BGE-reranker-large re-ranking module built on XLM-RoBERTa to refine candidate knowledge fragments by relevance score. The Domain + GTE–DPR retrieval setup achieves consistent MRR metrics across test trials, outperforming general-purpose DPR and domain-only DPR alternatives in semantic matching precision for industrial text.

After filtered and ranked knowledge chunks are combined with original user prompts via prompt engineering, two open-source LLMs—ChatGLM2-6B and LLaMA2-7B—generate domain-grounded answers anchored to validated industrial documents rather than generic model pre-training data. Controlled experiments contrast standalone LLM inference against the full modular RAG system, with ablation analysis isolating performance gains brought separately by the question classifier and dense retrieval sub-systems. The paper notes the framework still faces constraints from incomplete capture of niche industrial terminology during text preprocessing and limited alignment between traditional automatic evaluation metrics and human expert judgment of answer quality, and outlines future directions including iterative knowledge base expansion and adoption of specialized AI response assessment tools for industrial QA validation.

The paper “Knowledge-Enhanced Industrial Question-Answering Using Large Language Models,” is authored by Ronghui Liu, Hao Ren, Haojie Ren, Wu Rui, Wei Cui, Xiaojun Liang, Chunhua Yang, Weihua Gui. Full text of the open access paper: https://doi.org/10.1016/j.eng.2025.07.035. For more information about Engineering, visit the website at https://www.sciencedirect.com/journal/engineering.