TelecomGPT: A Framework to Build Telecom-Specific Large Language Models
Paper: TelecomGPT: A Framework to Build Telecom-Specific Large Language Models Authors: Hang Zou, Qiyang Zhao, Yu Tian, Lina Bariah, Faouzi Bader, Thierry Lestable, Merouane Debbah TL;DR — We built a three-stage pipeline (continual pre-training → instruction tuning → alignment) to adapt open-source LLMs for telecom. Key results: 75.3% on 3GPP document classification — nearly 2x GPT-4o’s 38.9% Outperforms GPT-4 on telecom math equation reconstruction (49.45 vs 49.38 MathBERT score) 4x improvement on telecom code infilling over base Llama3-8B-Instruct All built on 7-8B parameter models — a fraction of GPT-4’s size and cost Motivation Large Language Models like GPT-4 and Llama-3 are impressive generalists, but they struggle with the telecom domain. Ask GPT-4 to classify a 3GPP technical specification into the correct working group, and it gets it right less than 40% of the time. Ask it to infill a missing equation in a wireless communications paper, and it barely outperforms a coin flip. ...