Getting Started with Ollama: Running AI Models Made Easy

What is Ollama?

Ollama is a platform that enables seamless interaction with Open-Source Large Language Models (LLMs). By providing an API, Ollama empowers developers and users to harness the power of these advanced language models from the comfort of their own terminals or through simple API integration. With Ollama, you can tap into the collective intelligence of various LLMs, each with its unique strengths and capabilities.

You can find these models on the Ollama library https://ollama.com/library

This article will cover how we can install the Ollama server and run different models on our local machine.

Install Ollama

Get started with Ollama by installing the server on your local machine or running it in a Docker container.

Local Installation

To install Ollama locally, follow the simple steps on their official download page. This will give you direct access to the Ollama API and allow you to experiment with different models.

Docker Container

Alternatively, you can run Ollama in a Docker container. This blog post provides a step-by-step guide to get started.

Run an LLM Model

To run an LLM model with Ollama, you'll need to execute the Ollama Command Line Interface (CLI). The command will vary depending on how you installed Ollama.

Local Installation

If you installed Ollama locally on your machine, simply type ollama in your terminal to access the CLI.

Docker Container

If you choose to install Ollama using a Docker container, use the following command: docker exec -it ollama ollama. This command runs the ollama CLI within the container created during installation.

After mentioning the Ollama CLI, I will simply mention ollama and you will need to adapt for our case, the local CLI using a Docker container.

Pull model

Before running a model, it's necessary to pull a model. This process is similar to running a Docker container, where first you need to pull a Docker image.

To do that you can execute:

ollama pull llama2

or

ollama run llama2

The first command will simply pull the model to our local machine, while the second command will both pull the model and execute it, similar to if you were to run docker run hello-world.

List models

After pulling models for your local machine, you can list the models using:

ollama list

You will see information for all models you have pulled, such as:

NAME            	ID          	SIZE  	MODIFIED      
orca-mini:latest	2dbd9f439647	2.0 GB	9 seconds ago	

If you want to view more detailed information about a specific model, you can use:

ollama show orca-mini --modelfile

This command will display the following information:

# Modelfile generated by "ollama show"
# To build a new Modelfile based on this, replace FROM with:
# FROM orca-mini:latest

FROM /root/.ollama/models/blobs/sha256-66002b78c70a22ab25e16cc9a1736c6cc6335398c7312e3eb33db202350afe66
TEMPLATE "{{- if .System }}
### System:
{{ .System }}
{{- end }}

### User:
{{ .Prompt }}

### Response:
"
SYSTEM You are an AI assistant that follows instruction extremely well. Help as much as you can.
PARAMETER stop ### System:
PARAMETER stop ### User:
PARAMETER stop ### Response:

Don't worry about the details of this information for now; we will cover it in a future article.

Run Model

To run a model, you can simply execute:

ollama run orca-mini

After running the command, you can interact with the model through the console or using the API.

When using the console, you will see:

>>> Send a message (/? for help)

Type whatever you want, and the model will respond accordingly:

>>> Hello there
 Hello! How can I assist you today?

>>> Send a message (/? for help)

Alternatively, you can use the API to interact with the model:

curl -X POST <http://localhost:11434/api/generate> -d '{"model": "orca-mini", "prompt": "Hello There"}'

This will return a JSON response with the model's output:

{"model":"orca-mini","created_at":"2024-05-19T15:09:35.080692215Z","response":" Hello","done":false}
{"model":"orca-mini","created_at":"2024-05-19T15:09:35.205445963Z","response":"!","done":false}
{"model":"orca-mini","created_at":"2024-05-19T15:09:35.317453748Z","response":" How","done":false}
{"model":"orca-mini","created_at":"2024-05-19T15:09:35.434430239Z","response":" can","done":false}
{"model":"orca-mini","created_at":"2024-05-19T15:09:35.546670587Z","response":" I","done":false}
{"model":"orca-mini","created_at":"2024-05-19T15:09:35.680359625Z","response":" assist","done":false}
{"model":"orca-mini","created_at":"2024-05-19T15:09:35.811461808Z","response":" you","done":false}
{"model":"orca-mini","created_at":"2024-05-19T15:09:35.921968711Z","response":" today","done":false}
{"model":"orca-mini","created_at":"2024-05-19T15:09:36.05192137Z","response":"?","done":false}
{"model":"orca-mini","created_at":"2024-05-19T15:09:36.232287478Z","response":"","done":true,"done_reason":"stop","context":[31822,13,8458,31922,3244,31871,13,3838,397,363,7421,8825,342,5243,10389,5164,828,31843,9530,362,988,362,365,473,31843,13,13,8458,31922,9779,31871,13,27903,1091,13,13,8458,31922,13166,31871,13,16644,31905,1035,473,312,2803,365,1703,31902],"total_duration":1656297515,"load_duration":3944201,"prompt_eval_count":9,"prompt_eval_duration":455834000,"eval_count":10,"eval_duration":1151568000}

To see information about the running models, you can run:

ollama ps

NAME         	ID          	SIZE  	PROCESSOR      	UNTIL              
llama3:latest	a6990ed6be41	5.4 GB	54%/46% CPU/GPU	4 minutes from now

Replace orca-mini with the name of the model you want to run.

Conclusion

In this article, we have explored the basics of running models using Ollama, an open-source AI platform. We started by pulling and listing available models and then demonstrated how to run a model using the ollama run command. We also showed how to interact with the model through the console or API, and provided examples of the JSON response format.

With Ollama, you can easily experiment with different models and explore their capabilities. Whether you're a developer looking to integrate AI into your application or simply want to learn more about AI, Ollama provides a powerful toolset for exploring the possibilities of artificial intelligence.

We hope this article has provided a helpful introduction to Ollama and running models using this platform.