Proliferation of the electronics GPTs
Various industries are developing their own large language models for specific engineering applications.
While AI has been used for medical systems for several years, and ChatGPT has been used for chip design, connection with Wolfram analysis and even robot control, LLMs and GPTs are now being used for telecoms, optoelectronics and even power grid designs.
TelecomGPT, OptoGPT and ‘GridGPT’ have all been developed using LLMs to help engineers.
OptoGPT
OptoGPT has been developed by University of Michigan engineers and uses the computer architecture underpinning ChatGPT to work backward from desired optical properties to the material structure that can provide them.
The new algorithm designs optical multilayer film structures—stacked thin layers of different materials—that can serve a variety of purposes. Well-designed multilayer structures can maximize light absorption in a solar cell or optimize reflection in a telescope. They can improve semiconductor manufacturing with extreme UV light, and make buildings better at regulating heat with smart windows that become more transparent or more reflective depending on temperature.
OptoGPT produces designs for multilayer film structures within 0.1 seconds, almost instantaneously. In addition, OptoGPT’s designs contain six fewer layers on average compared to previous models, meaning its designs are easier to manufacture.
“Designing these structures usually requires extensive training and expertise as identifying the best combination of materials, and the thickness of each layer, is not an easy task,” said L. Jay Guo, professor of electrical and computer engineering at the University of Michigan.
To automate the design process for optical structures, the research team tailored a transformer architecture to treat materials at a certain thickness as words, also encoding their associated optical properties as inputs. Seeking out correlations between these “words,” the model predicts the next word to create a “phrase”—in this case a design for an optical multilayer film structure—that achieves the desired property such as high reflection.
“In a sense, we created artificial sentences to fit the existing model structure,” said Guo.
Researchers tested the new model’s performance using a validation dataset containing 1,000 known design structures including their material composition, thickness and optical properties. When comparing OptoGPT’s designs to the validation set, the difference between the two was only 2.58%, lower than the closest optical properties in the training dataset at 2.96%.
Taking analysis a step further, the researchers used a statistical technique to map out associations that OptoGPT makes.
“The high-dimensional data structure of neural networks is a hidden space, too abstract to understand. We tried to poke a hole in the black box to see what was going on,” Guo said.
When mapped in a 2D space, materials cluster by type such as metals and dielectric materials, which are electrically insulating but can support an internal electric field. All dielectrics, including semiconductors, converge upon a central point as the thickness approaches 10 nanometers. From an optics perspective, the pattern makes sense as light behaves similarly regardless of material as they approach such small thicknesses, helping further validate OptoGPT’s accuracy.
Known as an inverse design algorithm because it starts with the desired effect and works backward to a material design, OptoGPT offers more flexibility than previous inverse design algorithm approaches, which were developed for specific tasks. It enables researchers and engineers to design optical multilayer film structures for a wide breadth of applications.
GridGPT
A study by Na Li, Professor of Electrical Engineering and Applied Mathematics at the Harvard John A. Paulson School of Engineering and Applied Sciences (SEAS) suggests that LLMs could play an important role in co-managing some aspects of the grid, including emergency and outage response, crew assignments and wildfire preparedness and prevention.
But security and safety concerns need to be addressed before LLMs and GPTs can be deployed in the field.
“There is so much hype with large-language models, it’s important for us to ask what LLMs can do well and, perhaps more importantly, what they can’t do well, at least not yet, in the power sector,” said Le Xie, Professor of Electrical & Computer Engineering at Texas A&M University. “The best way to describe the potential of LLMs in this sector is as a co-pilot. It’s not a pilot yet — but it can provide advice, a second opinion, and very timely responses with very few training data samples, which is really beneficial to human decision making.”
The research team, which included engineers from Houston-based energy-provider CenterPoint Energy and grid-operator Midcontinent Independent System Operator, used GPT models to explore the capabilities of LLMs in the energy sector and identified both strengths and weaknesses.
But there are significant challenges to implementing LLMs in the energy sector — not the least of which is the lack of grid-specific data to train the models. For obvious security reasons, crucial data about the US power system is not publicly available and cannot be made public.
Another issue is the lack of safety guardrails. The power grid, like autonomous vehicles, needs to prioritize safety and incorporate large safety margin when making real-time decisions. LLMs and GPTs also need to get better about providing reliable solutions and transparency around their uncertainties, said Li.
“We want foundational LLMs to be able to say ‘I don’t know’ or ‘I only have 50% certainty about this response’, rather than give us an answer that might be wrong,” said Li. “We need to be able to count on these models to provide us with reliable solutions that meet specified standards for safety and resiliency.”
“As engineers, we want to highlight these limitations because we want to see how we can improve them,” said Li. “Power system engineers can help improve security and safety guarantees by either fine tuning the foundational LLM or developing our own foundational model for the power systems. One exciting part of this research is that it is a snapshot in time. Next year or even sooner, we can go back and revisit all these challenges and see if there has been any improvement.”
TelecomGPT
A framework for building LLMs for 6G telecoms systems has been developed at Khalifa University in Abu Dhabi. Current mainstream LLMs generally lack the specialized knowledge in telecom domain. Researchers have proposes a pipeline to adapt any general purpose LLMs to a telecom-specific LLMs by collecting telecom-specific pretrain dataset, instruction dataset, preference dataset to perform continual pre-training, instruct tuning and alignment tuning respectively.
They developed three benchmarks, Telecom Math Modeling, Telecom Open QnA and Telecom Code Tasks. These provide an evaluation of the capabilities of LLMs and GPTs including math modeling, Open-Ended question answering, code generation, infilling, summarization and analysis in telecom domain.
The resulting fine-tuned LLM TelecomGPT outperforms state of the art LLMs including GPT-4, Llama-3 and Mistral in Telecom Math Modeling benchmark significantly and achieve comparable performance in various evaluation benchmarks such as TeleQnA, 3GPP technical documents classification, telecom code summary and generation and infilling.
www.seas.harvard.edu; www.umich.edu; www.ku.ac.ae
If you enjoyed this article, you will like the following ones: don't miss them by subscribing to :
