China Trained a 1-Trillion-Parameter LLM Using Only Domestic Chips

China Telecom has reportedly developed two large language models using domestically manufactured AI chips, a significant milestone in China's pursuit of semiconductor independence. This achievement is crucial as the US is tightening its ban on supplying high-end chips to China. Huawei, a leader in China's local chip industry, is believed to have played a key role in this development. If China Telecom's models were trained using Huawei chips alone, it would be a massive success for Huawei and the Chinese government. The project's GitHub page hints at the use of Huawei's Ascend Atlas 800T A2 training server, which supports Kunpeng 920 processors. These processors are built using the Arm 8.2 architecture and have a 7nm process. One of the models has 1 trillion parameters, while the other, TeleChat2t-115B, has over 100 billion parameters. This development is a significant step forward for China's semiconductor industry. China's goal is to become independent of other countries for its semiconductor needs. The US-China chip war has led to increased restrictions on chip supplies to China, making this achievement even more notable.