Image created by Author using Midjourney
Large language models (LLMs) have become extremely prominent and useful for all sorts of tasks, but new users may find the large number of LLM tools and utilities intimidating. This article focuses on 5 of the available and widely-useful such tools, all of which are no-cost and created to help maturing minds take advantage of the wide variety of available language models: Transformers, LlamaIndex, Langchain, Ollama, and Llamafile.
1. Transformers
One of the most prominent libraries for modern natural language processing (NLP) model frameworks, Transformers comes from the NLP powerhouse Hugging Face. The variety of pre-trained models available in Transformers is vast, with both foundational and fine-tuned models designed for tasks such as text classification, translation, question answering, and more.
Key Features
- versatility (models exist for backends like PyTorch and TensorFlow)
- plentiful pre-trained models that can be customized
- user-friendly APIs and docs
- a robust user base to answer questions and help
Transformers is good for new users, as it is very simple to pick up the basics, but also useful enough to help with even the most complex of tasks. The library comes with extensive documentation, user-friendly APIs, and a nearly-unfathomable collection of available models. With Transformers, beginners can start using state-of-the-art models without a ton of deep learning knowledge.
Getting Started
First, install Transformers:
Example: Loading a pre-trained model and running inference
|
from transformers import pipeline
classifier = pipeline(“sentiment-analysis”) result = classifier(“I love using the Transformers library!”) print(result) |
2. LlamaIndex
LlamaIndex is a data framework customized for LLM use cases, especially retrieval augmented generation (RAG). It streamlines connections between LLMs and different data sources, thus enabling the easy building of complicated data-based LLM applications.
Key Features
- built-in basic data source connectors
- ability to customize for different use cases and complexity levels
- a variety of pre-packaged task-specific starter solutions in the form of Llama Packs
- ample documentation
LlamaIndex is helpful for beginners because it simplifies the initial setup and takes care of the plumbing required to connect data to application, allowing for easy integration with data sources as well as tinkering to one’s liking. Thanks to its solid documentation, developers can quickly pick up what they need to get going and build their applications in a particular direction.
Getting Started
First, install the library:
Example: Building a very simple RAG application (taken from here)
Note that for this example your OpenAI API key must be set as an environment variable, and that LlamaIndex uses OpenAI’s gpt-3.5-turbo model by default. I am also using a copy of the Machine Learning Mastery ebook “Maximizing Productivity with ChatGPT” as my sole RAG document, located in the “data” directory.
|
from llama_index.core import VectorStoreIndex, SimpleDirectoryReader
documents = SimpleDirectoryReader(“data”).load_data() index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine() response = query_engine.query(“Tell me a little about prompt engineering.”) print(response) |
3. Langchain
LangChain is a framework which allows AI engineers to connect language models with a vast array of data sources, as well as with other LLMs. Langchain also provides pathways for context-aware reasoning applications, offering tools for building capable agents that can perform complex reasoning tasks for problem solving.
Key Features
- an interface for creating and handling agents, tools and libraries
- and support for reasoning applications and the tracing and evaluation thereof
Beginners can use Langchain to quickly build intelligent agents, as it makes application development painless and comes with a robust set of tools and templates to get things moving.
Getting Started
Install Langchain via pip:
Example: Check out the Langchain quickstart guide for a useful introductory tutorial
4. Ollama
Ollama is designed to provide easy access to multiple LLMs, such as Llama 3, Mistral, Gemma and more, and makes managing them painless by lessening both deployment and management overhead. You can use Ollama to quickly setup local LLMs for both interaction as well as development.
Key Features
- support for multiple large language models
- integration with a range of libraries and extensions
- painless methodologies to deploy models
Ollama is good for beginners since it brings together a slew of leading large language models, and makes them easier to deploy and run. Get your hands on Llama 3 locally, for example, and then connect to the same model via Ollama in your favorite LLM development framework (Langchain, LlamaIndex, etc.) for development. It really solves multiple problems at once.
Getting Started
Install Ollama via their website for your platform, and then use the Python library to interact:
Example: Use a model in your own Python application (taken from here)
|
import ollama
response = ollama.generate(model=‘gemma:2b’, prompt=‘what is a qubit?’) print(response[‘response’]) |
5. Llamafile
Llamafile was born to make sharing and running LLMs a cinch with a single file. It makes distributing and running models painless by keeping its process simple and straightforward.
Key Features
- one-click sharing and running of LLMs
- incredibly easy setup and use
- variable backend support
This tool helps manage LLM assets, which in turn assist with communicating with and running LLMs. Its minimal complexity gives additional ease to newbies.
Getting Started
Use pip to install Llamafile:
Example: load and query the Mistral llamafile from the command line
|
./mistral–7b–instruct–v0.2.Q5_K_M.llamafile —temp 0.3 –p ‘[INST]what is a qubit?[/INST]’ |
In this article, we have outlined 5 tools to get beginners started using LLMs: Transformers, LlamaIndex, Langchain, Ollama, and Llamafile. Each one offers a unique set of tasks, advantages, and features, aimed at aiding beginners in grasping the subtleties of the LLM development landscape, and interacting with it. These tools provide a great jumping off point for understanding LLMs.
Be sure to visit and reference each of the project’s repositories and documentation to help guide you in your quest to learn and experiment with these tools. Enjoy the process!