In November 2022, ChatGPT achieved significant success by attracting 1 million users within five days of its launch. The potential value of generative AI tools is substantial and has the ability to transform entire industries. Venture capitalists invested $2.6 billion in generative AI startups in 2022.
However, there is a steep cost associated with the potential value of generative AI tools. The more complex the generative model, the more expensive the required compute resources. The running cost of ChatGPT is $100,000 per day, while Google’s competitor, Bard, reportedly costs the company ten times more per query than a standard keyword search. Additionally, the environmental costs of excessive computer consumption will be significant.
The high costs of training such a complex model, the learning curve to understand new mathematical formulations, and the need to develop specialized expertise in the tooling and infrastructure add to the costs of generative AI. Consequently, these costs may deter enterprise adoption of generative AI, despite its wide-ranging use cases, from generating marketing content to writing code. To truly enable the generative AI revolution for enterprise applications, the costs need to be significantly reduced.
Quantum-inspired techniques, which draw inspiration from quantum physics, have the potential to reduce the computational costs associated with large language models (LLMs). Moreover, these quantum-inspired techniques could broaden the use cases of generative AI for enterprises, particularly in solving complex optimization problems.
How Quantum Techniques Could Potentially Bring Down The Costs Of Generative AI
Tensor networks are efficient linear algebraic structures for representing complex correlations between variables that originated from quantum physics. Initially used by quantum physicists to simulate quantum states on classical computers, tensor networks could potentially play a critical role in the development of quantum computing. Tensor networks can simulate quantum circuits on classical hardware today and may be replaced with real quantum circuits in the future.
This allows users to develop “quantum applications” on classical hardware that can run on the fault-tolerant quantum computers of the future. Tensor networks could also have near-term business value for generative AI by reducing the size and speeding up large neural networks, including transformers, the NN type used in ChatGPT, through “tensorization.”
Through tensorization, tensor networks have the potential to reduce both the costs and carbon footprint of generative AI, making generative models more accessible and accelerating their adoption in the enterprise. This cost reduction could also enable generative AI to be deployed on edge devices such as phones and voice assistants. Tensorization could also reduce the costs of other complex models, such as Monte Carlo simulations, which can take up a significant portion of a finance enterprise’s entire compute budget.
Tensor networks can compress generative models, and the resulting tensorized generative models can also generate higher-quality samples. This has significant implications for enterprise applications of generative AI beyond classic text and image generation. Tensorized generative models could generate novel solutions to complex optimization problems prevalent in industrial settings.
Enhancing Optimization Through Generative Modeling
Optimization problems are those where the aim is to find the best solution from a range of possibilities, based on certain criteria. In industrial settings, optimization problems can involve balancing thousands of conflicting goals, such as maximizing production output while minimizing costs.
One approach to solving such problems is to use a generative model trained on the best existing solutions, which can then generate new solutions. This technique is called generator-enhanced optimization (GEO), although it has yet to be peer-reviewed.
While any generative model can be used for GEO, tensor networks have shown promise in generalizing from training data better than other classical models. This means that tensor networks can understand the essential features that make good solutions, and generate new ones rather than repeating trained solutions.
The structure of tensor networks also aids in solving optimization problems. For instance, equality constraints can be encoded directly into tensor networks, ensuring that only valid samples are output. This is an improvement over traditional methods, which often struggle with equality constraints, and the tensorized approach could deliver better performance while using fewer computational resources since the tensor network can be parameterized more sparsely with more equality constraints.
Limitations Of Tensor Networks For Generative AI
As with any emerging technology, there will be challenges to overcome when it comes to implementing tensor networks and generative modeling in the enterprise. One major challenge will be translating complex business problems into mathematical formulations that can be effectively solved using these techniques. There will also be a learning curve involved in understanding the mathematical principles behind tensor networks and how they can be used as an alternative data structure to neural networks.
In addition, teams will need to develop expertise in the tooling and infrastructure required to make these solutions work. This will involve developing robust workflows that can integrate new data from across the enterprise. Finally, the outputs of these models will need to be translated into insights that can inform business decisions, which will require multifunctional teams with specialized expertise and the right technology tools to be successful.
Gaining More Value From Generative AI
The potential impact of tensor networks can be amplified if they are eventually replaced by real quantum circuits running on fault-tolerant quantum devices. By using tensor networks, enterprises can benefit from generative AI today and prepare for value creation in the quantum computing era.
As quantum hardware continues to mature, it is important to consider which hardware configurations could provide the most value for building on tensor network-based generative models.
