CentOS Stream 10
Sponsored Link

Ollama : Install2025/02/13

 

Install Ollama that is an application which allows you to run LLM locally.

It runs reasonably fast even on computers without a GPU.

[1] Install Ollama.
[root@dlp ~]#
wget https://ollama.ai/install.sh

[root@dlp ~]#
sh ./install.sh

>>> Installing ollama to /usr/local
>>> Downloading Linux amd64 bundle
################################################################################### 100.0%
>>> Creating ollama user...
>>> Adding ollama user to render group...
>>> Adding ollama user to video group...
>>> Adding current user to ollama group...
>>> Creating ollama systemd service...
>>> Enabling and starting ollama service...
Created symlink '/etc/systemd/system/default.target.wants/ollama.service' → '/etc/systemd/system/ollama.service'.
>>> The Ollama API is now available at 127.0.0.1:11434.
>>> Install complete. Run "ollama" from the command line.
WARNING: No NVIDIA/AMD GPU detected. Ollama will run in CPU-only mode.

# ollama service started

[root@dlp ~]#
systemctl status ollama

● ollama.service - Ollama Service
     Loaded: loaded (/etc/systemd/system/ollama.service; enabled; preset: disabled)
     Active: active (running) since Thu 2025-02-13 08:46:30 JST; 1min 44s ago
 Invocation: cffbe6ef806a42c6934771e107573866
   Main PID: 2007 (ollama)
      Tasks: 8 (limit: 100433)
     Memory: 13.1M (peak: 13.5M)
        CPU: 21ms
     CGroup: /system.slice/ollama.service
.....
.....

# ollama listens port 11434 on localhost

[root@dlp ~]#
ss -napt | grep ollama

LISTEN 0      4096       127.0.0.1:11434      0.0.0.0:*    users:(("ollama",pid=2007,fd=3))
[2] If you like to access it from other hosts, you can change the range it listens on.
[root@dlp ~]#
systemctl edit ollama.service
# add [OLLAMA_HOST] variable
# to listen all, specify [0.0.0.0]
# to listen internal network, specify internal IP address

[Service]
Environment="OLLAMA_HOST=0.0.0.0"

[root@dlp ~]#
systemctl restart ollama.service
[root@dlp ~]#
ss -napt | grep ollama

LISTEN 0      4096               *:11434            *:*    users:(("ollama",pid=2073,fd=3))
[3] To use Ollama, download models first.
# example below is 8B model from llama3

[root@dlp ~]#
ollama pull llama3:8b

pulling manifest
pulling 6a0746a1ec1a... 100% ▏ 4.7 GB
pulling 4fa551d4f938... 100% ▏  12 KB
pulling 8ab4849b038c... 100% ▏  254 B
pulling 577073ffcc6c... 100% ▏  110 B
pulling 3f8eb4da87fa... 100% ▏  485 B
verifying sha256 digest
writing manifest
success

# example below is phi4 model from Microsoft

[root@dlp ~]#
ollama pull phi4

pulling manifest
pulling fd7b6731c33c... 100%  9.1 GB
pulling 32695b892af8... 100%   275 B
pulling fa8235e5b48f... 100%  1.1 KB
pulling 45a1c652dddc... 100%    82 B
pulling f5d6f49c6477... 100%   486 B
verifying sha256 digest
writing manifest
success

# example below is Gemma2 model from Google

[root@dlp ~]#
ollama pull gemma2:2b

pulling manifest
pulling 7462734796d6... 100% ▏ 1.6 GB
pulling e0a42594d802... 100% ▏  358 B
pulling 097a36493f71... 100% ▏ 8.4 KB
pulling 2490e7468436... 100% ▏   65 B
pulling e18ad7af7efb... 100% ▏  487 B
verifying sha256 digest
writing manifest
success

# example below is Granite3 model from IBM

root@dlp:~#
ollama pull granite3-dense:8b

pulling manifest
pulling 1a9a38833607... 100% ▏ 4.9 GB
pulling 604785e698e9... 100% ▏  664 B
pulling 492069a62c25... 100% ▏  11 KB
pulling 31ad3abb7dce... 100% ▏   63 B
pulling d77201a4f430... 100% ▏  417 B
verifying sha256 digest
writing manifest
success
[4] How to use Ollama.
[root@dlp ~]#
ollama run llama3:8b

>>> Send a message (/? for help)
>>> Who are you?
I am LLaMA, an AI assistant developed by Meta AI that can understand and respond to human input in a conversational
manner. I'm not a human, but rather a computer program designed to simulate conversation, answer questions, and provide
information on a wide range of topics.

I was trained on a massive dataset of text from the internet and can generate responses that are natural-sounding and
often indistinguishable from those written by a human. My training data includes a vast array of texts from various
sources, including books, articles, and websites, which allows me to provide information on many different topics.

I'm constantly learning and improving my language abilities through the interactions I have with users like you. So,
please feel free to ask me anything, and I'll do my best to provide a helpful and accurate response!

>>> What are you good at?
I'm designed to be a conversational AI, so I'm good at:

1. **Answering questions**: I can process natural language queries and provide relevant answers from my training data.
2. **Generating text**: I can create human-like text responses to prompts or questions, making me suitable for
applications like chatbots, language translation, and content generation.
3. **Conversing**: I'm designed to engage in conversations, using context and understanding to respond to user input.
4. **Providing information**: I've been trained on a vast amount of text data, so I can provide information on various
topics, including history, science, technology, literature, and more.
5. **Summarizing content**: I can summarize long pieces of text into shorter, more digestible versions, highlighting key
points and main ideas.
6. **Generating creative writing**: I can create stories, poems, or dialogues based on prompts or topics.
7. **Understanding humor**: I can recognize and respond to humorous language, including puns, sarcasm, and irony.
8. **Adapting to tone**: I can adjust my responses to match the tone of the user's input, whether it's formal, informal,
serious, or playful.
9. **Learning from feedback**: Through user interactions, I can learn and improve my understanding of language and
generate more accurate and helpful responses over time.

Keep in mind that while I'm a highly advanced AI, there are still limitations to what I can do. For example:

* I may not always understand the nuances of human emotions or cultural context.
* My training data might be biased or outdated, which could impact my accuracy on certain topics.
* I'm not capable of physical actions or direct interaction with the physical world.

However, within the realm of digital communication and information exchange, I'm designed to be a powerful and helpful
tool!

to exit console, input [/bye]
>>> /bye

[root@dlp ~]#
ollama run phi4

>>> Who are you?
I'm Phi, a language model AI developed by Microsoft. I can assist with answering questions, providing explanations on
various topics, and offering guidance across many subjects. My purpose is to help users find information and understand
complex ideas as clearly as possible. If you have any specific questions or need assistance, feel free to ask!

>>> What are you good at?
I'm designed to assist with a wide range of tasks, including:

1. **Providing Information**: Offering explanations and answers on diverse topics such as science, history, technology,
culture, and more.

2. **Language Assistance**: Helping with grammar, spelling, punctuation, and writing style improvements.

3. **Problem-Solving**: Assisting with mathematical problems or logical reasoning challenges.

4. **Guidance and Advice**: Providing general advice on topics like productivity, time management, and wellness, though
it's important to consult professionals for specific personal advice.

5. **Educational Support**: Aiding in learning new subjects by explaining concepts clearly and offering study tips.

6. **Entertainment**: Engaging with users through trivia, jokes, or storytelling.

While I can be quite helpful, there are limitations too. I don't have access to real-time information or personal data
unless shared during the conversation, and my knowledge is up to date only until October 2023. For specialized or highly
specific inquiries, it's always a good idea to consult an expert in that field.

>>> /bye

[root@dlp ~]#
ollama run gemma2:2b

>>> Who are you?
I am Gemma, an AI assistant. I'm a large language model created by the Gemma team at Google DeepMind.  Think of me as a
computer program that can understand and generate human-like text.

I don't have real-world experience or personal opinions, and I can only communicate in text form.  Is there anything
specific you'd like to talk about today?

>>> What are you good at?
That's a great question!  I can do many things with language.  Here are some examples:

**Creative:**
* **Writing stories and poems:** Give me a prompt or an idea, and I can come up with something imaginative!
* **Creating different kinds of text formats:**  Need a script for a play, a song lyric, or even just a fun haiku?  I can
do it.

**Informational:**
* **Answering your questions:**  Whether it's about history, science, or pop culture - I try to give you accurate and
helpful answers!
* **Summarizing text:** Give me a long article and I can condense it into the key points.
* **Translating languages:** I'm still learning, but I can translate between many languages.

**Other:**
* **Brainstorming ideas:** Stuck on a project?  I can help you generate new ideas!
* **Playing language games:** I enjoy wordplay and fun challenges!


What are you interested in exploring today? 

>>> /bye
[5] How to use Ollama via API.
[root@dlp ~]# curl http://localhost:11434/api/chat -d '{
  "model": "gemma2:2b",
  "stream": false,
  "messages": [
    { "role": "user", "content": "What is the date today?" }
  ]
}'
{"model":"gemma2:2b","created_at":"2025-02-13T00:23:22.198141881Z","message":{"role":"assistant","content":"Today's date is **October 26, 2023**.   \n"},"done_reason":"stop","done":true,"total_duration":1292273847,"load_duration":28690600,"prompt_eval_count":15,"prompt_eval_duration":138000000,"eval_count":22,"eval_duration":1124000000}
# * the answer is wrong

[root@dlp ~]# curl http://dlp.srv.world:11434/api/chat -d '{
  "model": "phi4",
  "stream": false,
  "messages": [
    { "role": "user", "content": "Tell me the recommended sightseeing spots in Hiroshima, Japan." }
  ]
}'
{"model":"phi4","created_at":"2025-02-13T00:30:19.35567879Z","message":{"role":"assistant","content":"Hiroshima is a city rich in history and culture, offering numerous sightseeing opportunities. Here are some of the most recommended spots:\n\n1. **Hiroshima Peace Memorial Park**: \n   - Home to the iconic *Genbaku Dome* (A-Bomb Dome), which was preserved as a symbol of peace.\n   - The park also includes the Hiroshima Peace Memorial Museum, which provides an in-depth look at the atomic bombing and its aftermath.\n\n2. **Hiroshima Castle**:\n   - A beautiful reconstructed castle offering panoramic views of the city from its tower.\n   - Explore the well-preserved gardens and samurai armor collection inside.\n\n3. **Itsukushima Shrine (Miyajima)**:\n   - Although technically outside Hiroshima, it’s a short ferry ride away to Miyajima Island.\n   - Famous for its \"floating\" torii gate at high tide and scenic hiking trails.\n\n4. **Shukkeien Garden**:\n   - A traditional Japanese garden with picturesque landscapes, small bridges, and waterfalls.\n   - Ideal for a peaceful stroll surrounded by nature.\n\n5. **Hiroshima Museum of Art**:\n   - Located in the Hiroshima Prefectural Art Museum Building, it hosts both modern and contemporary art exhibitions.\n   - The building itself is an architectural marvel worth visiting.\n\n6. **Nagisa Pier \u0026 Hiroshima Beach**:\n   - A lively area perfect for a leisurely walk, shopping, or dining by the sea.\n   - Offers great views of the Seto Inland Sea.\n\n7. **Hiroshima City Asyuku Museum (As you like it)**:\n   - Known for its quirky and modern architecture and exhibitions on art, design, and technology.\n   - A perfect spot for those interested in contemporary culture.\n\n8. **Miyajima Taisha Shrine**:\n   - Famous for the deer that roam freely around the area.\n   - Offers beautiful views of the island's landscapes.\n\n9. **Hiroshima Shukkei-en Garden**:\n   - Known as Japan’s oldest Western-style garden, designed to reflect European castle gardens.\n   - Features a lake, miniature mountains, and traditional Japanese buildings.\n\n10. **Ebisu Shrine**:\n    - A tranquil shrine nestled in a natural forest, known for its serene atmosphere and seasonal beauty.\n\nThese sites collectively provide insights into Hiroshima's resilience, culture, and history while offering diverse experiences from peaceful gardens to engaging museums."},"done_reason":"stop","done":true,"total_duration":121119090882,"load_duration":11037524,"prompt_eval_count":23,"prompt_eval_duration":243000000,"eval_count":503,"eval_duration":120863000000}
Matched Content