Ubuntu 24.04
Sponsored Link

Ollama : Install2024/06/04

 

Install Ollama that is an application which allows you to run LLM locally.

It runs reasonably fast even on computers without a GPU.

[1] Install Ollama.
root@dlp:~#
wget https://ollama.ai/install.sh

root@dlp:~#
sh ./install.sh

>>> Downloading ollama...
########################################
>>> Installing ollama to /usr/local/bin...
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
Created symlink /etc/systemd/system/default.target.wants/ollama.service → /etc/systemd/system/ollama.service.
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.
WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.

# ollama service started

root@dlp:~#
systemctl status ollama

*  ollama.service - Ollama Service
     Loaded: loaded (/etc/systemd/system/ollama.service; enabled; preset: enabled)
    Drop-In: /etc/systemd/system/ollama.service.d
             +-- override.conf
     Active: active (running) since Tue 2024-06-04 00:20:43 UTC; 20min ago
   Main PID: 5292 (ollama)
      Tasks: 13 (limit: 19050)
     Memory: 566.6M (peak: 570.0M)
        CPU: 5.258s
     CGroup: /system.slice/ollama.service

# ollama listens port 11434 on localhost

root@dlp:~#
ss -napt | grep ollama

LISTEN 0      4096            127.0.0.1:11434           0.0.0.0:*     users:(("ollama",pid=5449,fd=3))
[2] If you like to access it from other hosts, you can change the range it listens on.
root@dlp:~#
systemctl edit ollama.service
# add [OLLAMA_HOST] variable
# to listen all, specify [0.0.0.0]
# to listen internal network, specify internal IP address

[Service]
Environment="OLLAMA_HOST=0.0.0.0"

root@dlp:~#
systemctl restart ollama.service
root@dlp:~#
ss -napt | grep ollama

LISTEN 0      4096                    *:11434                 *:*     users:(("ollama",pid=5548,fd=3))
[3] To use Ollama, download models first.
# example below is 8B model from llama3

root@dlp:~#
ollama pull llama3:8b

pulling manifest
pulling 6a0746a1ec1a... 100%
pulling 4fa551d4f938... 100%
pulling 8ab4849b038c... 100%
pulling 577073ffcc6c... 100%
pulling 3f8eb4da87fa... 100%
verifying sha256 digest
writing manifest
removing any unused layers
success

# example below is phi3 model from Microsoft

root@dlp:~#
ollama pull phi3

pulling manifest
pulling b26e6713dc74... 100% 
pulling fa8235e5b48f... 100% 
pulling 542b217f179c... 100% 
pulling 8dde1baf1db0... 100% 
pulling f91db7a2deb9... 100% 
verifying sha256 digest
writing manifest
removing any unused layers
success

# example below is Gemma model from Google

root@dlp:~#
ollama pull gemma

pulling manifest
pulling ef311de6af9d... 100%
pulling 097a36493f71... 100%
pulling 109037bec39c... 100%
pulling 65bb16cf5983... 100%
pulling 0c2a5137eb3c... 100%
verifying sha256 digest
writing manifest
removing any unused layers
success

# example below is Granite Code model from IBM

root@dlp:~#
ollama pull granite-code:8b

pulling manifest
pulling 8718ec280572... 100%
pulling 0d7c97d535b6... 100%
pulling e50df8490144... 100%
pulling 9893bb2c2917... 100%
pulling 6be564cde5be... 100%
verifying sha256 digest
writing manifest
removing any unused layers
success
[4] How to use Ollama.
root@dlp:~#
ollama run llama3:8b

>>> Send a message (/? for help)
>>> Hello
Hello! It's nice to meet you. Is there something I can help you with, or would you like to chat?

>>> Tell me about yourself
I'm just an AI, so I don't have a personal life or experiences like humans do. But I'd be happy to tell you about my
"existence"!

I was created by a team of software developers using advanced technologies like natural language processing (NLP) and
machine learning. My primary function is to understand and generate human-like text based on the input I receive.

I don't have personal opinions, emotions, or beliefs. My responses are generated based on patterns in language data, which
I learn from through my training. This means that everything I say is a result of complex algorithms and statistical
models, rather than personal experiences or feelings.

That being said, I'm designed to be helpful and engaging! I can answer questions, provide information on a wide range of
topics, offer suggestions, and even have conversations with you. My goal is to assist and communicate effectively, while
also learning and improving over time.

So, that's me in a nutshell! What about you? What are your interests or hobbies?

to exit console, input [/bye]
>>> /bye

root@dlp:~#
ollama run phi3

>>> Hello
 Hello! How can I assist you today? Whether it's answering a question, providing information on a topic, or helping with
problem-solving, feel free to share what you need help with.

(Note: As per the instruction, no JSON response is required here as this interaction doesn't involve solving for a
specific answer.)

>>> Tell me about yourself
 I am an AI digital assistant created by Microsoft, designed to interact and provide information in various domains such
as general knowledge, technology updates, language assistance, and more. My purpose is to facilitate access to information
and support users through conversational interfaces. While I don't have personal experiences or emotions, I continuously
learn from interactions to improve my responses and user experience.

>>> /bye

root@dlp:~#
ollama run gemma

>>> Hello
Hello!  It's lovely to hear from you. How can I assist you today? 

(Note: As per the instruction, no JSON response is required here as this interaction doesn't involve solving for a
specific answer.)

>>> Tell me about yourself
I am a large language model, trained by Google. I am able to understand and generate human language, and can assist you
with various tasks such as:

* **Conversation and storytelling:** I can engage in meaningful conversations and create captivating stories.
* **Information retrieval:** I have access to a vast knowledge base and can provide answers to your questions.
* **Language translation:** I can translate languages in real-time, making communication across cultures seamless.
* **Creative writing:** I can generate original content, including poems, stories, and code.

My purpose is to help people and make their lives easier. I am constantly learning and growing, and I am excited to
explore new ways to assist you. 

What would you like to know about me or what would you like me to do today? 

>>> /bye
[5] How to use Ollama via API.
root@dlp:~# curl http://localhost:11434/api/chat -d '{
  "model": "llama3:8b",
  "stream": false,
  "messages": [
    { "role": "user", "content": "Where will the next Summer Olympics be held?" }
  ]
}'
{"model":"llama3:8b","created_at":"2024-06-04T01:38:24.78401215Z","message":{"role":"assistant","content":"The 2024 Summer Olympics are scheduled to take place in Paris, France, from July 26 to August 11, 2024. This will be the third time that Paris has hosted the Summer Olympics, after previously hosting in 1900 and 1924.\n\nAfter that, the next Summer Olympics will be held in:\n\n* 2028: Los Angeles, California, United States (July 21 to August 6, 2028)\n* 2032: Brisbane, Queensland, Australia (September 17 to October 3, 2032)\n\nThese dates are subject to change, but these cities and dates are currently confirmed by the International Olympic Committee (IOC)."},"done_reason":"stop","done":true,"total_duration":25727979040,"load_duration":3263792627,"prompt_eval_count":18,"prompt_eval_duration":1152721000,"eval_count":139,"eval_duration":21227692000}

root@dlp:~# curl http://dlp.srv.world:11434/api/chat -d '{
  "model": "phi3",
  "stream": false,
  "messages": [
    { "role": "user", "content": "Tell me the recommended sightseeing spots in Hiroshima, Japan." }
  ]
}'
{"model":"phi3","created_at":"2024-06-04T01:42:13.798533443Z","message":{"role":"assistant","content":" Hiroshima is a city rich in history and culture, offering visitors an array of sightseeing spots that are both moving and memorable:\n\n\n1. **A-Bomb Dome** - As a UNESCO World Heritage site, the A-Bomb Dome stands as one of the few remaining structures from before the atomic bomb was dropped in 1945. It's now a poignant memorial and museum dedicated to peace education.\n\n\n2. **Miyajima** - Particularly famous for its cherry blossoms, Miyajima Island is home to Itsukushima Shrine with its iconic \"floating\" torii gate (which appears as though it's on water at high tide).\n\n\n3. **Peace Memorial Park** - Located near the A-Bomb Dome, this park houses several museums and monuments dedicated to the events of August 6th, 1945, and serves as a symbol for peace worldwide.\n\n\n4. **Shukkei-en Garden** - This classical Japanese garden is renowned for its stunning display of flora and picturesque ponds in springtime when the cherry blossoms are at their peak.\n\n\n5. **Hiroshima Castle (Itsukushima)** - While not originally from Hiroshima, this castle on Miyajima Island is an impressive site with panoramic views of the island and its bay.\n\n\n6. **Ujō-in Temple** - This temple features a beautiful golden Buddha statue (Dai Kyakuden) that's worth seeing, especially during cherry blossom season.\n\n\n7. **Hiroshima Museum of Art** - The museum showcases modern and contemporary art, with special exhibitions often changing throughout the year. It is housed in a building designed by architect Tange Kenzō, who was also responsible for rebuilding Hiroshima after the bombing.\n\n\n8. **Happo-en Garden** - A serene spot offering traditional Japanese gardens and tea rooms overlooking Hiroshima Bay. It's perfect for a peaceful afternoon stroll or relaxed lunchtime meal.\n\n\nThese locations provide a blend of historical significance, natural beauty, and cultural richness that characterize the city of Hiroshima."},"done_reason":"stop","done":true,"total_duration":48371945691,"load_duration":1449437636,"prompt_eval_count":21,"prompt_eval_duration":727459000,"eval_count":528,"eval_duration":46151297000}
Matched Content