Ollama Llama3

Posted on | In

Source

[Running Ollama on Google Colab (Free Tier): A Step-by-Step Guide by Anoop Maurya May, 2024 Medium](https://medium.com/@mauryaanoop3/running-ollama-on-google-colab-free-tier-a-step-by-step-guide-9ef74b1f8f7a)

Introduction

Colab Ollama Llama3

Ref: [Running Ollama on Google Colab (Free Tier): A Step-by-Step Guide by Anoop Maurya May, 2024 Medium](https://medium.com/@mauryaanoop3/running-ollama-on-google-colab-free-tier-a-step-by-step-guide-9ef74b1f8f7a)

Step 1: Installing package and loading the extension:

1
2
!pip install colab-xterm #https://pypi.org/project/colab-xterm/
%load_ext colabxterm

Step 2: Opening Terminal

1
%xterm

在 terminal window

1
2
curl -fsSL https://ollama.com/install.sh | sh
ollama serve & ollama pull llama3

有非常多的 models!

image-20240616231117021

使用 Langchain

1
2
3
4
5
6
7
8
9
10
11
!pip install langchain-community 

# Import Ollama module from Langchain
from langchain_community.llms import Ollama

# Initialize an instance of the Ollama model
llm = Ollama(model="llama2")

# Invoke the model to generate responses
response = llm.invoke("Tell me a joke")
print(response)

使用 Ollama API

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
from importlib.metadata import version

pkgs = ["tqdm",    # Progress bar
       ]

for p in pkgs:
    print(f"{p} version: {version(p)}")
    
    
import urllib.request
import json

def query_model(prompt, model="llama3", url="http://localhost:11434/api/chat"):
    # Create the data payload as a dictionary
    data = {
        "model": model,
        "seed":123,        # for deterministic responses
        "temperature":0,   # for deterministic responses
        "messages": [
            {"role": "user", "content": prompt}
        ]
    }

    # Convert the dictionary to a JSON formatted string and encode it to bytes
    payload = json.dumps(data).encode("utf-8")

    # Create a request object, setting the method to POST and adding necessary headers
    request = urllib.request.Request(url, data=payload, method="POST")
    request.add_header("Content-Type", "application/json")

    # Send the request and capture the response
    response_data = ""
    with urllib.request.urlopen(request) as response:
        # Read and decode the response
        while True:
            line = response.readline().decode("utf-8")
            if not line:
                break
            response_json = json.loads(line)
            response_data += response_json["message"]["content"]

    return response_data


result = query_model("What do Llamas eat?")
print(result)