ETRI Launches Ambitious Project for 100B-Parameter AI Model

The Electronics and Telecommunications Research Institute (ETRI) of South Korea has embarked on an ambitious initiative to develop a 100 billion parameters artificial intelligence (AI) foundation model. This project is part of the national strategy to enhance the country’s AI capabilities and secure technological independence. ETRI was selected as one of five research teams in the NC AI Consortium, supported by the Ministry of Science and ICT, and began extensive research in August.

Strengthening South Korea’s AI Sovereignty

ETRI’s involvement in the NC AI Consortium marks a significant step in establishing itself as a key player in the AI research community. The project, titled “Development of a Scalable Multimodal Generative Foundation Model for Industrial AI Transition,” aims to create AI technologies that can seamlessly integrate and process various types of data, including text, voice, and images. This initiative is designed to bolster the application of AI across major sectors such as manufacturing, healthcare, education, and culture.

As part of its preparations, ETRI has built a solid research infrastructure, leveraging support from the National Information Society Agency (NIA) for data and from the National IT Industry Promotion Agency (NIPA) for GPU resources. This collaboration is crucial for the effective pre-training of the 100B-scale model, positioning South Korea toward greater technological autonomy in the development of large-scale AI systems.

Advancing Multimodal AI Technologies

The core objective of this project is to create a multimodal AI model that can effectively process diverse data formats. ETRI’s Intelligence Information Research Division is integrating technologies developed from previous national projects into this new research. Notably, the Language Intelligence Research Section is applying insights from the language model “Eagle,” which has been designed to enhance understanding and inference capabilities in the Korean language.

Additionally, the Embodied Integrated Intelligence Research Section is focusing on creating multimodal models that emphasize audio and video capabilities. This is based on prior research achievements, including technologies for assessing brain function decline. The Visual Intelligence Research Section is also working on improving vision-language fusion performance, drawing on technologies developed through models like “KOALA” and “Ko-LLaVA.”

According to Oh-woog Kwon, assistant vice president of ETRI’s Intelligence Information Research Division and project manager, “ETRI has successfully navigated initial resource challenges and is making steady progress in pre-training the 100B-scale model. With close collaboration with NC AI, we aim to lead the transformation of industrial AI through the development of a scalable and reliable multimodal foundation model.”

This initiative is expected to play a pivotal role in South Korea’s goal of becoming a global leader in AI technology. By fostering innovation and intelligent transformation across various industries, ETRI is positioning itself as a cornerstone of the country’s AI ecosystem.

In summary, the development of the 100B-scale AI foundation model not only aims to enhance South Korea’s AI capabilities but also reflects a broader strategy to establish independent technological advancements in the field. With ongoing support and research, ETRI is set to make significant strides in the global AI landscape.