AI stock valuations are sky high (S&P 500âs rise is controlled by 4 stocks and 1/2 the recent gains are from Nvidia)⦠I remember just starting in computer hardware around Y2K, and at the time, every company in the NASDAQ would need to outperform Microsoft to justify their valuation. Clearly not sustainable. Current AI stock valuations are potentially a replay of the dotcom bust, and you might think we are headed for AI disappointment. However, the future of AI is bright and I want to share my view on how it will roll-out to be the revolution people believe but canât conceptually visualize.
Historically, and we ONLY have to go back to 2021 when LLMs didnât really exist as a thing the population could interact with. In the summer of 2021 I talked about the rise of only a few companies being able to build LLMs and also the subsequent future of AI coming to your smart device. We wanted neurothink to be the hardware/software solutions to give a much larger population access to this knowledge and compute power. We got close with a working 4 Petaflop solution virtualized for hundreds of users at once but without the foundation of base models to retrain in a guided ML way.
A short three years later, LLMs that take hundreds of petaflops and >10,000 H100 GPUs now exist and we can call them ground truth despite the hallucinations. Those can be accessed directly in the cloud, and this aggregated knowledge has spawned SLM (Small Language Models) that learn from LLMs either directly or indirectly and are usually available from places like huggingface.
Weekly, I am getting questions about how to get started with AI or the best AI architecture for my company. Lets look at fiveway you and your company can take advantage of AI today:
1. Direct access of a LLM in the cloud today. These can help with your ad-hoc work and you get the power of the most powerful collection of general knowledge humanity has created so far. The downside is that you pay ongoing costs, your data is not private and it is harder to integrate into a repeatable business process.
2. You can pseudo-customize ChatGPT today with either a customGPT or their API function (to build a custom app etc..). This allows you to âinstructâ the LLM and also influence the response based on specific documents you provide and thus makes a more repeatable business tool for your team. However, you still pay the subscription fees and your data is shared in the cloud.
3. You can move to an agent based system like langchain and that allows you to call the LLM but gives you more repeatable control of the behavior, retention of the history of the conversation (improving responses) and private RAG functionality (retrieval augmented generation) which lets you use the LLM as the language operating system and your files which you encode give company specific responses. You can also build a process to prevent Prompt Hacking by taking the response from the LLM and forcing a review against rules or behaviors your company wants to enforce (i.e. dont hurt humans, make a bomb etc..), and you can do this for multiple uses-cases or client types (agents).
4. You can retrain the model with a concept called LoRA (low rank adaption) and that gives you a customized LLM based on the main LLM in the cloud. This also means that your data and custom training are in the cloud, but you gain quicker and more specific responses to your unique business requests.
5. You can re-train or do transfer learning on a model in-house. The benefits are that there is no subscription to a LLM. You can choose a model specific for your task (for our company we tested 3 vision categorization models and picked the best one in just 1 week), you can train the model frequently based on business updates, you can put multiple models together (ensemble models: first determine the category of the request, then have another model handle a specific task etc..), and finally your data and model are private and yours. The downside is that you need to manage your baby and your babysitter (data scientist) can be a bit expensive but those can be your choices of how far to push the model and how frequently to update.
We are really coming close to the day when you can take an app or AI code and just swap the LLM and swap the compute without retraining or new code. Some of the new technology is coming from OpenAI and also Intel OneAPI to abstract your Python code from your GPU instruction set (CUDA, ROCm or Maia etc..).
In summary, the AI revolution got started with hardware and software capability and cool demonstrations (moving robots, lifelike images from text, passing the SAT etc..) but this is not where the value of AI will come from. This abstraction of the model from the hardware and the ability to customize and âcompressâ large models is what will jump-start a business process revolution with AI.
To get started personally, and test the waters with ZERO ongoing costs, here are the ingredients you need:
- compute. I personally use a laptop with enough GPU or shared memory (Mac M3 max with 96GB).
- Some basic software. I use conda to manage my python versions and then I use Lamma.cpp which allows me to run my own SLM on my laptop. There is also ollamma and LMStudio if you want a more graphical interface.
- A model. I go to huggingface and use a 32B parameter model, but new models are coming from everyone Meta, Apple, Microsoft etc..
- some basic terminal commands to get it up and running. In fact, I can handle 4+ users in the background and it is accessible in my house by my family and with my remote team for my company.
- do a quick youtube search on hosting your own LLM and there are some very good 15 minute videos to get you up and running.