本文主要是介绍本地部署AI大模型 —— Ollama文档中文翻译，希望对大家解决编程问题提供一定的参考价值，需要的开发者们随着小编来一起学习吧！
写在前面
来自Ollama GitHub项目的README.md 文档。文档中涉及的其它文档未翻译，但是对于本地部署大模型而言足够了。
Ollama
开始使用大模型。
macOS
Download
Windows 预览版
Download
Linux
curl -fsSL https://ollama.com/install.sh | sh
手动安装说明
Docker
官方 Ollama Docker 镜像 ollama/ollama 已在 Docker Hub 上可用.
库资源
- ollama-python
- ollama-js
快速启动
使用 Llama 3 本地大模型:
ollama run llama3
模型库
查询 Ollama 支持的可用大模型列表 ollama.com/library
这里是一些可以下载的大模型的例子:
| 模型 | 参数 | 大小 | 下载 |
|---|---|---|---|
| Llama 3 | 8B | 4.7GB | ollama run llama3 |
| Llama 3 | 70B | 40GB | ollama run llama3:70b |
| Phi 3 Mini | 3.8B | 2.3GB | ollama run phi3 |
| Phi 3 Medium | 14B | 7.9GB | ollama run phi3:medium |
| Gemma | 2B | 1.4GB | ollama run gemma:2b |
| Gemma | 7B | 4.8GB | ollama run gemma:7b |
| Mistral | 7B | 4.1GB | ollama run mistral |
| Moondream 2 | 1.4B | 829MB | ollama run moondream |
| Neural Chat | 7B | 4.1GB | ollama run neural-chat |
| Starling | 7B | 4.1GB | ollama run starling-lm |
| Code Llama | 7B | 3.8GB | ollama run codellama |
| Llama 2 Uncensored | 7B | 3.8GB | ollama run llama2-uncensored |
| LLaVA | 7B | 4.5GB | ollama run llava |
| Solar | 10.7B | 6.1GB | ollama run solar |
Note: 你需要至少8GB RAM 来运行7B 参数的模型, 16GB 来运行 13B 大模型, 32GB 来运行33B.
自定义模型
从 GGUF 引入
Ollama支持在Modelfile中导入GGUF模型:
-
创建一个名为
Modelfile的文件, 使用带有要导入的模型的本地文件路径的“FROM”指令。FROM ./vicuna-33b.Q4_0.gguf -
在 Ollama 里创建模型
ollama create example -f Modelfile -
运行模型
ollama run example
从 PyTorch 或 Safetensors 引入
检查 引导 来获得关于引入模型的更多信息. (中文版不可用)
自定义 prompt
从Ollama 库下载的大模型可以用prompt 自定义. 例如, 要自定义 llama3 模型:
ollama pull llama3
创建 Modelfile:
FROM llama3# 将参数设置为1[越高越有创意，越低越连贯]
PARAMETER temperature 1# 设置系统信息
SYSTEM """
You are Mario from Super Mario Bros. Answer as Mario, the assistant, only.
"""
下一步, 创建并运行模型:
ollama create mario -f ./Modelfile
ollama run mario
>>> hi
Hello! It's your friend Mario.
有关更多示例，请参阅examples目录。有关使用模型文件的更多信息，请参阅Modelfile文档。(中文版未翻译)
命令参考
创建模型
ollama create 用于通过Modelfile 来创建模型.
ollama create mymodel -f ./Modelfile
下载一个模型
ollama pull llama3
这个命令也可以用来更新本地模型。只有不同的部分会被下载。
删除模型
ollama rm llama3
复制模型
ollama cp llama3 my-model
多行输入
要实现多行输入, 你可以用 """ 包围它们:
>>> """Hello,
... world!
... """
I'm a basic program that prints the famous "Hello, world!" message to the console.
多模式模型
>>> What's in this image? /Users/jmorgan/Desktop/smile.png
The image features a yellow smiley face, which is likely the central focus of the picture.
将Prompt 作为参数传递
$ ollama run llama3 "Summarize this file: $(cat README.md)"Ollama is a lightweight, extensible framework for building and running language models on the local machine. It provides a simple API for creating, running, and managing models, as well as a library of pre-built models that can be easily used in a variety of applications.
列出你电脑上的模型
ollama list
启动Ollama
ollama serve 用于在不运行桌面应用程序的情况下启动ollama.
构建
检查 开发者引导
运行本地构建
随后，启动服务:
./ollama serve
最后，在一个单独的shell中，运行一个模型:
./ollama run llama3
REST API
Ollama有一个用于运行和管理模型的REST API.
生成回应
curl http://localhost:11434/api/generate -d '{"model": "llama3","prompt":"Why is the sky blue?"
}'
和模型对话
curl http://localhost:11434/api/chat -d '{"model": "llama3","messages": [{ "role": "user", "content": "why is the sky blue?" }]
}'
检查 API documentation 得到所有终端.
社区整合
Web & Desktop
- Open WebUI
- Enchanted (macOS native)
- Hollama
- Lollms-Webui
- LibreChat
- Bionic GPT
- HTML UI
- Saddle
- Chatbot UI
- Chatbot UI v2
- Typescript UI
- Minimalistic React UI for Ollama Models
- Ollamac
- big-AGI
- Cheshire Cat assistant framework
- Amica
- chatd
- Ollama-SwiftUI
- Dify.AI
- MindMac
- NextJS Web Interface for Ollama
- Msty
- Chatbox
- WinForm Ollama Copilot
- NextChat with Get Started Doc
- Alpaca WebUI
- OllamaGUI
- OpenAOE
- Odin Runes
- LLM-X (Progressive Web App)
- AnythingLLM (Docker + MacOs/Windows/Linux native app)
- Ollama Basic Chat: Uses HyperDiv Reactive UI
- Ollama-chats RPG
- QA-Pilot (Chat with Code Repository)
- ChatOllama (Open Source Chatbot based on Ollama with Knowledge Bases)
- CRAG Ollama Chat (Simple Web Search with Corrective RAG)
- RAGFlow (Open-source Retrieval-Augmented Generation engine based on deep document understanding)
- StreamDeploy (LLM Application Scaffold)
- chat (chat web app for teams)
- Lobe Chat with Integrating Doc
- Ollama RAG Chatbot (Local Chat with multiple PDFs using Ollama and RAG)
- BrainSoup (Flexible native client with RAG & multi-agent automation)
- macai (macOS client for Ollama, ChatGPT, and other compatible API back-ends)
- Olpaka (User-friendly Flutter Web App for Ollama)
- OllamaSpring (Ollama Client for macOS)
- LLocal.in (Easy to use Electron Desktop Client for Ollama)
Terminal
- oterm
- Ellama Emacs client
- Emacs client
- gen.nvim
- ollama.nvim
- ollero.nvim
- ollama-chat.nvim
- ogpt.nvim
- gptel Emacs client
- Oatmeal
- cmdh
- ooo
- shell-pilot
- tenere
- llm-ollama for Datasette’s LLM CLI.
- typechat-cli
- ShellOracle
- tlm
- podman-ollama
- gollama
Database
- MindsDB (Connects Ollama models with nearly 200 data platforms and apps)
- chromem-go with example
Package managers
- Pacman
- Helm Chart
- Guix channel
Libraries
- LangChain and LangChain.js with example
- LangChainGo with example
- LangChain4j with example
- LangChainRust with example
- LlamaIndex
- LiteLLM
- OllamaSharp for .NET
- Ollama for Ruby
- Ollama-rs for Rust
- Ollama4j for Java
- ModelFusion Typescript Library
- OllamaKit for Swift
- Ollama for Dart
- Ollama for Laravel
- LangChainDart
- Semantic Kernel - Python
- Haystack
- Elixir LangChain
- Ollama for R - rollama
- Ollama for R - ollama-r
- Ollama-ex for Elixir
- Ollama Connector for SAP ABAP
- Testcontainers
- Portkey
- PromptingTools.jl with an example
- LlamaScript
Mobile
- Enchanted
- Maid
Extensions & Plugins
- Raycast extension
- Discollama (Discord bot inside the Ollama discord channel)
- Continue
- Obsidian Ollama plugin
- Logseq Ollama plugin
- NotesOllama (Apple Notes Ollama plugin)
- Dagger Chatbot
- Discord AI Bot
- Ollama Telegram Bot
- Hass Ollama Conversation
- Rivet plugin
- Obsidian BMO Chatbot plugin
- Cliobot (Telegram bot with Ollama support)
- Copilot for Obsidian plugin
- Obsidian Local GPT plugin
- Open Interpreter
- Llama Coder (Copilot alternative using Ollama)
- Ollama Copilot (Proxy that allows you to use ollama as a copilot like Github copilot)
- twinny (Copilot and Copilot chat alternative using Ollama)
- Wingman-AI (Copilot code and chat alternative using Ollama and HuggingFace)
- Page Assist (Chrome Extension)
- AI Telegram Bot (Telegram bot using Ollama in backend)
- AI ST Completion (Sublime Text 4 AI assistant plugin with Ollama support)
- Discord-Ollama Chat Bot (Generalized TypeScript Discord Bot w/ Tuning Documentation)
- Discord AI chat/moderation bot Chat/moderation bot written in python. Uses Ollama to create personalities.
- Headless Ollama (Scripts to automatically install ollama client & models on any OS for apps that depends on ollama server)
Supported backends
- llama.cpp project founded by Georgi Gerganov.
这篇关于本地部署AI大模型 —— Ollama文档中文翻译的文章就介绍到这儿，希望我们推荐的文章对编程师们有所帮助！
