環境
準備
TheBloke/TinyLlama-1.1B-Chat-v1.0-GGUFのtinyllama-1.1b-chat-v1.0.Q4_K_M.ggufを ダウンロードします
llama.cppのディレクトリのルート直下に入れます
実行
まどマギテスト
推論
./main -ngl 35 -m tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf --temp 0.1 -n -1 -p "<|system|>\nあなたはアニメ評論家です</s>\n<|user|>\nまどマギで一番可愛いキャラを教えてください</s>\n<|assistant|>"
結果
<|system|>\nあなたはアニメ評論家です\n<|user|>\nまどマギで一番可愛いキャラを教えてください\n<|assistant|>|\nSure, here's a list of the most adorable characters from Mao Maki: \n - 1. Yuki - A cute and playful fox who loves to explore and has a big heart. - 2. Kuro - A mysterious and mischievous cat with a sharp tongue and a love for adventure. - 3. Momo - A cheerful and energetic rabbit who loves to play and is always up for a good time. - 4. Sora - A curious and inquisitive bird who loves to explore and has a big heart. - 5. Rin - A kind and gentle dragon who loves to help others and has a big heart. - 6. Tsuki - A wise and wise owl who loves to teach and is always up for a good adventure. - 7. Kyou - A playful and curious monkey who loves to explore and has a big heart. - 8. Matsu - A gentle and kind rabbit who loves to play and has a big heart. - 9. Yuki - A cute and playful fox who loves to explore and has a big heart. - 10. Kuro - A mysterious and mischievous cat with a sharp tongue and a love for adventure. [end of text] llama_print_timings: load time = 68.47 ms llama_print_timings: sample time = 74.67 ms / 302 runs ( 0.25 ms per token, 4044.41 tokens per second) llama_print_timings: prompt eval time = 1305.44 ms / 60 tokens ( 21.76 ms per token, 45.96 tokens per second) llama_print_timings: eval time = 36928.61 ms / 301 runs ( 122.69 ms per token, 8.15 tokens per second) llama_print_timings: total time = 38412.24 ms
富士山の高さは？
推論
./main -ngl 35 -m tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf --temp 0.1 -n -1 -p "<|system|>\nあなたは優秀なAIアシスタントです</s>\n<|user|>\n富士山の高さは？</s>\n<|assistant|>"
結果
<|system|>\nあなたは優秀なAIアシスタントです\n<|user|>\n富士山の高さは？\n<|assistant|> The height of Fuji-san, the highest mountain in Japan, is 3,776 meters (12,380 feet). [end of text] llama_print_timings: load time = 78.75 ms llama_print_timings: sample time = 8.47 ms / 33 runs ( 0.26 ms per token, 3895.18 tokens per second) llama_print_timings: prompt eval time = 999.13 ms / 51 tokens ( 19.59 ms per token, 51.04 tokens per second) llama_print_timings: eval time = 4288.58 ms / 32 runs ( 134.02 ms per token, 7.46 tokens per second) llama_print_timings: total time = 5307.22 ms
What American movies do you recommend that make you cry?
推論
./main -ngl 35 -m tinyllama-1.1b-chat-v1.0.Q4_K_M.gguf --temp 0.1 -n -1 -p "<|system|>\nYou are an excellent AI assistant.</s>\n<|user|>\nWhat American movies do you recommend that make you cry?</s>\n<|assistant|>"
結果
<|system|>\nYou are an excellent AI assistant.\n<|user|>\nWhat American movies do you recommend that make you cry?\n<|assistant|>Sure, here are some American movies that make me cry: 1. "Schindler's List" (1993) - This film tells the story of a German businessman who saves over 1,000 Jews during World War II and is later honored by the Nazis for his heroism. The emotional scenes between Oskar Schindler and his Jewish workers are particularly powerful. 2. "The Notebook" (2004) - This romantic drama tells the story of a young couple in the 1940s who fall in love despite their different social classes. The film's heartbreaking ending is one of its most memorable moments. 3. "Pride and Prejudice" (2005) - This adaptation of Jane Austen's classic novel tells the story of Elizabeth Bennet, a young woman who must navigate societal expectations and love amidst her family's opposition. The film's emotional scenes between Elizabeth and Mr. Darcy are particularly memorable. 4. "The Favourite" (2018) - This period drama tells the story of Queen Anne, who is forced to choose between two potential husbands in the early 1700s. The film's complex characters and emotional scenes make it a standout among its genre. 5. "The Revenant" (2015) - This action-adventure film tells the story of Hugh Glass, a frontiersman who is hired to rescue his family from an attack by Native Americans. The film's heartbreaking scenes between Glass and his son are particularly memorable. I hope these recommendations help you find some new films to watch that make you cry! [end of text] llama_print_timings: load time = 58.66 ms llama_print_timings: sample time = 102.49 ms / 380 runs ( 0.27 ms per token, 3707.57 tokens per second) llama_print_timings: prompt eval time = 994.55 ms / 45 tokens ( 22.10 ms per token, 45.25 tokens per second) llama_print_timings: eval time = 45669.67 ms / 379 runs ( 120.50 ms per token, 8.30 tokens per second) llama_print_timings: total time = 46898.81 ms
