Importing Open Source Models to Ollama

Clone from huggingface.co:

 

apt install git-lfs

git clone https://huggingface.co/nvidia/Llama-3.1-Nemotron-70B-Instruct-HF

 

 

Build GGUF

 

in the git dir:

docker run --rm -v .:./model ollama/quantize -q q4_K_M /model

 

 

Import GGUD to ollama

# Modelfile
FROM "./Llama-3.1-Nemotron-70B-Instruct-HF.gguf"
PARAMETER stop "<|im_start|>"
PARAMETER stop "<|im_end|>"
TEMPLATE """
<|im_start|>system
{{ .System }}<|im_end|>
<|im_start|>user
{{ .Prompt }}<|im_end|>
<|im_start|>assistant
"""

# Create and run model
ollama create capy-hermes -f Modelfile
ollama run capy-hermes