仿ŽãªããããŒã«ã«LLMãGPUã§åãããWSL2ã
ã²ãŒãã³ã°PCã§LLM
Dockerã§OllamaãšOpen WebUI ã䜿ã£ãŠ
ããŒã«ã«ã§LLMãåãããŠã¿ãŸãã
ç°å¢
- Windows11
CPU Intel(R) Core(TM) i7-9700 CPU @ 3.00GHz
RAM 32.0 GB
GPU NVIDIA GeForce RTX 2060
å°çš GPU ã¡ã¢ãª 6.0 GB
å ±æ GPU ã¡ã¢ãª 16 GB - WSL2
Ubuntu 22.04.3 LTS
çµ±åçãã€ã¯ã©ã®ã¬ã€ãã¬ãŒã·ã³ã°æäœèŠä»¶ã®GPUã§ã
WSL2ã«Dockerãã€ã³ã¹ããŒã«æžã¿ã§ã
ã¡ã¢ãªã«ã€ããŠ
ãã©ã¡ãŒã¿æ°ã®å€§ããªã¢ãã«ãåããã«ã¯ãååãªã¡ã¢ãªãå¿
èŠã«ãªããŸã
çè
ã®ç°å¢ã§ã¯ããã©ã¡ãŒã¿æ°7BçšåºŠã®ã¢ãã«ãå©çšã§ããŸãã
Note: You should have at least 8 GB of RAM available to run the 7B models, 16 GB to run the 13B models, and 32 GB to run the 33B models.
Dockerã§Ollamaãèµ·åãã
Ollamaã¯LLM(Large Language Model å€§èŠæš¡èšèªã¢ãã«)ãããŒã«ã«ã§ç°¡åã«åãããããŒã«ã§ã
macãšLinuxã«å¯Ÿå¿ãwindowsã¯èšäºæçš¿ææç¹ã§ã¯ãã¬ãã¥ãŒçã®ã¿ãããŸã
ä»åã¯WSLäžã®Dockerã«æ§ç¯ããŸã
ãåèã«ãGPU察å¿ã®Ollamaã³ã³ãããèµ·åããŸã
WSL2ã®Ubuntuã«NVIDIA Container Toolkitãã€ã³ã¹ããŒã«ããŸã
curl -fsSL https://nvidia.github.io/libnvidia-container/gpgkey \
| sudo gpg --dearmor -o /usr/share/keyrings/nvidia-container-toolkit-keyring.gpg
curl -s -L https://nvidia.github.io/libnvidia-container/stable/deb/nvidia-container-toolkit.list \
| sed 's#deb https://#deb [signed-by=/usr/share/keyrings/nvidia-container-toolkit-keyring.gpg] https://#g' \
| sudo tee /etc/apt/sources.list.d/nvidia-container-toolkit.list
sudo apt-get update
sudo apt-get install -y nvidia-container-toolkit
Dockerã§äœ¿ããããã«èšå®ããŸã
sudo nvidia-ctk runtime configure --runtime=docker
sudo systemctl restart docker
GPUã䜿çšããŠã³ã³ãããèµ·åããŸã
docker run -d --gpus=all -v ollama:/root/.ollama -p 11434:11434 --name ollama ollama/ollama
Ollamaãããã©ã«ãã§ãµããŒãããŠããllama2ãšãããã©ã¡ãŒã¿æ°7Bã®ã¢ãã«ãããŠã³ããŒãããŠ
CLIã§æšè«ããŠã¿ãŸã
docker exec -it ollama ollama run llama2
3.8GBããã®ã§ããŠã³ããŒãã«ãã°ããããããŸã
å
¥åæ¬ã衚瀺ããããé©åœã«ããã³ãããæžããŸã
>>> why the sky is blue?
llama2ã®åç(é·ãã®ã§æãç³ã¿)
The sky appears blue to our eyes because of a phenomenon called
Rayleigh scattering. This is the scattering of sunlight by small
particles in the Earth's atmosphere, such as nitrogen and oxygen
molecules, and small dust particles. These particles scatter shorter
(blue) wavelengths of light more than longer (red) wavelengths,
causing the sky to appear blue.
The reason for this scattering is that the wavelength of light is
related to its frequency, and smaller particles scatter light with
higher frequencies (bluer light) more than larger particles. This is
known as Rayleigh's Law, which states that the amount of scattering is
inversely proportional to the fourth power of the wavelength of light.
The blue color of the sky can also be affected by other factors, such
as the presence of water vapor and pollutants in the atmosphere. For
example, when there is more water vapor in the air, it can scatter
light in all directions, causing the sky to appear more hazy and less
blue. Similarly, pollutants like dust and smoke can also scatter
light, giving the sky a yellow or brown tint.
In addition to Rayleigh scattering, the blue color of the sky can also
be influenced by the way our brains process visual information. The
human eye is wired to perceive blue as a more vibrant and intense
color than red, which means that the blue light scattered by the
atmosphere appears brighter and more vivid to us than the red light.
Overall, the blue color of the sky is a result of a combination of
physical processes in the atmosphere and the way our brains interpret
visual information.
google翻蚳
空ã¯ãªãéãã®ã§ãã?
ç§ãã¡ã®ç®ã«ç©ºãéãèŠããã®ã¯ãã空ãéãèŠããããšããçŸè±¡ã«ãããã®ã§ãã
ã¬ã€ãªãŒæ£ä¹±ã ããã¯å°ããªãã®ã«ãã倪éœå
ã®æ£ä¹±ã§ãã
å°çã®å€§æ°äžã®çªçŽ ãé
žçŽ ãªã©ã®ç²å
ååãå°ããªå¡µç²åã ãããã®ç²åã¯ããçãæ£ä¹±ããŸã
å
ã®æ³¢é·ãé·ãïŒèµ€ïŒããé·ãïŒéïŒã
空ãéãèŠããåå ãšãªããŸãã
ãã®æ£ä¹±ã®çç±ã¯ãå
ã®æ³¢é·ã
å
ã¯ãã®åšæ³¢æ°ã«é¢ä¿ããŠãããããå°ããç²åã¯å
ãæ£ä¹±ããŸãã
ããé«ãåšæ³¢æ°ïŒããéãå
ïŒã¯ããã倧ããªç²åãããå€ããªããŸãã ããã¯
ã¬ã€ãªãŒã®æ³åãšããŠç¥ããããæ£ä¹±ã®éã¯æ¬¡ã®ãšããã§ãã
å
ã®æ³¢é·ã®4ä¹ã«åæ¯äŸããŸãã
空ã®éè²ã¯ã次ã®ãããªä»ã®èŠå ã«ã圱é¿ãããããšããããŸãã
倧æ°äžã®æ°Žèžæ°ãæ±æç©è³ªã®ååšãªã©ã ã®ããã«
ããšãã°ã空æ°äžã«æ°Žèžæ°ãå€ããªããšã飿£ããå¯èœæ§ããããŸãã
ããããæ¹åã«å
ãåœãããšã空ãããæã£ãŠèŠãã«ãããªããŸãã
éã åæ§ã«ãç²å¡µãç
ãªã©ã®æ±æç©è³ªã飿£ããå¯èœæ§ããããŸãã
æããã空ã«é»è²ãŸãã¯è¶è²ã®è²åããäžããŸãã
ã¬ã€ãªãŒæ£ä¹±ã«å ããŠã空ã®éè²ããŸãã
ç§ãã¡ã®è³ãèŠèŠæ
å ±ãåŠçããæ¹æ³ã«åœ±é¿ãåããŸãã ã®
人éã®ç®ã¯ãéãããé®®ããã§åŒ·çãªãã®ãšããŠèªèããããã«ã§ããŠããŸãã
èµ€ãããè²ã匷ããšããããšã¯ãéãå
ãæ£ä¹±ããããšãæå³ããŸãã
ç§ãã¡ã«ã¯å€§æ°ã¯èµ€è²å
ãããæããé®®ããã«èŠããŸãã
å
šäœãšããŠã空ã®éãè²ã¯æ¬¡ã®çµã¿åããã®çµæã§ãã
倧æ°äžã®ç©ççããã»ã¹ãšè³ã®è§£éæ¹æ³
èŠèŠçãªæ
å ±ã
ããã«å¿çãå§ãŸã
ããã£ãœãæãã®åçãåŸãããŸãã
ç¥ãããã©
ãŸãæšè«äžã¯GPUã®ã¡ã¢ãªã5GBã»ã©äœ¿çšããŠããŸãã
çµäºã¯byeã§ã
>>> /bye
GUIããŒã«ã䜿ã
Ollama察å¿ã®ChatGPT颚ç»é¢ã®GUIããŒã«ãããã®ã§äœ¿ã£ãŠã¿ãŸã

Open WebUI
ãã¡ããDockerã§èµ·åããŸã
ããŒã«ã«ãã¹ãã«Ollamaãèµ·åæžã¿ã®å Žå
äžèšã§ã³ã³ãããèµ·åã§ããŸã
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui ghcr.io/open-webui/open-webui:main
windowsåŽãã©ãŠã¶ã§localhost:3000ã«ã¢ã¯ã»ã¹ãããš
ãã°ã€ã³ç»é¢ã衚瀺ãããŸã
é©åœã«ã¢ã«ãŠã³ããäœã£ãŠãã°ã€ã³ããŸã
ã¢ãã«ãéžãã§
ãã£ããæ¬ã«ããã³ãããæžããŸã

why the sea is blue?
google翻蚳
ãªãæµ·ã¯éãã®ã§ããïŒ
ãããã£ãŒã«
Llama2:ææ° 25/02/2024 08:23
æµ·ã¯æ¬¡ã®ãããªããŸããŸãªçç±ã§éãèŠããŸãã
倪éœå ã®åå°: ã¬ã€ãªãŒæ£ä¹±ãšåŒã°ããçŸè±¡ã«ãããæµ·ã¯å€ªéœããã®éè²å ãä»ã®è²ãããå€ãåå°ããŸãã ããã¯ãå ã®çãæ³¢é· (éã玫ãªã©) ã¯å šæ¹åã«æ£ä¹±ãããäžæ¹ãé·ãæ³¢é· (èµ€ããªã¬ã³ãžãªã©) ã¯æ°Žã«åžåãããæµ·ãéãèŠããããšãæå³ããŸãã
æ°Žã®åå: æ°Žã®ååã®æ§é ããæµ·ãéãèŠããçç±ã«é¢äžããŠããŸãã æ°Žã®ååã¯ç¬ç¹ã®åœ¢ç¶ãããŠãããèµ€ããªã¬ã³ãžãªã©ã®ç¹å®ã®æ³¢é·ã®å ãåžåããéã玫ã®å ãåå°ããŸãã ããã¯ã倪éœå ãæµ·ã«å ¥ããšãéãšçŽ«ã®å ãåå°ãããæµ·ãéãè²ã«ãªãããšãæå³ããŸãã
ç²åã®ãµã€ãº: æ°Žäžã®ç²åã®ãµã€ãºãæµ·ã®èŠãæ¹ã«åœ±é¿ãäžããå¯èœæ§ããããŸãã éæãªæ°Žã«å«ãŸãããããªå°ããªç²åã¯ã倧ããªç²åãããå ãæ£ä¹±ããåŸåããããæµ·ãããéãèŠããããšããããŸãã
æµ·ã®æ·±ã: æµ·ã®æ·±ãã¯ãæµ·ã®éãã®èŠãæ¹ã«ã圱é¿ããŸãã æµ·ã®å¥¥æ·±ãã«é²ããšãå ã¯æ°Žäžãããé·ãè·é¢ãç§»åããå¿ èŠããããããå ãæ£ä¹±ããŠããéãèŠããããšããããŸãã
ä»ã®ç©è³ªã®ååš: è»é¡ãå ç©ç©ãªã©ãæµ·æŽäžã®ä»ã®ç©è³ªããã®å€èгã«åœ±é¿ãäžããå¯èœæ§ããããŸãã ããšãã°ãæ°Žäžã«è»é¡ãå€éã«ååšãããšãæµ·ãç·ããã£ãè²åãã«ãªãããšããããŸãã
å šäœãšããŠãæµ·ã®éè²ã¯ãããã®èŠå ã®çµã¿åããã®çµæã§ãããæµ·ã®ç¹å®ã®å Žæãæ¡ä»¶ã«ãã£ãŠç°ãªããŸãã
æ¥æ¬èªå¯Ÿå¿ã¢ãã«ã䜿ã
Ollamaã«æ¥æ¬èªå¯Ÿå¿ã®ã¢ãã«ãã€ã³ã¹ããŒã«ããŠ
GUIããæšè«ãè¡ã£ãŠã¿ãŸã
Ollamaã®ã³ã³ããã«ã¢ã¿ããããŸã
docker exec -it ollama bash
Ollamaã®ããã©ã«ãã§ãµããŒããããŠããã¢ãã«ã«ã¯æ¥æ¬èªå¯Ÿå¿ãããŸããã®ã§
ggufãã¡ã€ã«ããã¢ãã«ãäœæããŸã
ã奜ã¿ã®æ¥æ¬èªå¯Ÿå¿ã¢ãã«ã®ggufãã¡ã€ã«ãããŠã³ããŒãããŸã
ãã¡ãã®ã¢ãã«ã䜿çšããŠã¿ãŸã
ãªã¹ã¹ã¡ã®æ¥æ¬èªå¯Ÿå¿ã¢ãã«ãããã°ã³ã¡ã³ãã§æããŠãã ãã
apt-get install wget
wget https://huggingface.co/TheBloke/Xwin-LM-13B-V0.1-GGUF/resolve/main/xwin-lm-13b-v0.1.Q2_K.gguf
Ollamaã§ã¢ãã«ãèªã¿èŸŒãããã«
ModelfileãäœæããŸã
echo "FROM ./xwin-lm-13b-v0.1.Q2_K.gguf" > Modelfile
ã¢ãã«ãäœæããŸã
xwinã®éšåã¯ä»»æã®ååãæå®ã§ããŸã
ollama create xwin -f Modelfile
åã³ãã©ãŠã¶ã§localhost:3000ã«ã¢ã¯ã»ã¹ãããš
远å ããã¢ãã«ãéžæã§ããããã«ãªã£ãŠããŸã

æ¥æ¬ã§äžçªé«ãå±±ã¯ïŒ
ãã©ã¡ãŒã¿æ°13Bãšå€§ãããããåçã®çæããã¡ããã¡ãéãã§ã
å±±ãšããæŠå¿µã«ã€ããŠããªãŒã¹ã¿ã€ã«ã§èªã£ãŠãããŸãã
ç¥èã¯ããã®ã§ãããã³ããã®å·¥å€«ãæ±ãããããã§ã
ããã³ãã工倫ããŸãã

æ¥æ¬ã§æãæšé«ãé«ãå±±ã®ååã¯ïŒ
è¬ã®å±±ãçæãããŠããŸãããã·ã³ãã«ã«ãªããŸãã
éãã®ã§ãããŸã§ã§ã
ã¡ãªã¿ã«ããæ¥ãã¹ã¯ãããã®å³äžã«çŸããCopilotã®åçã¯

ããŒã«ã«ã§ã¯ãªã
ã¢ãã«ãšãã©ã¡ãŒã¿æ°
ä»å䜿çšããã¢ãã«ã®ãã©ã¡ãŒã¿æ°ã¯äžèšã®éãã§ã
| ã¢ãã« | ãã©ã¡ãŒã¿æ° |
|---|---|
| Llama 2 | 7B |
| xwin-lm-13b-v0.1.Q2_K.gguf | 13B |
Bã¯billionã10å
ChatGPT3ã175B
GPT4ã1å
è¶
ããšèšãããŠããã®ã§
æ ŒãéããŸã
ChatGPTã¯ããŒã«ã«ã§åããŸããã
Discussion