LLM AI Agent toy project - #1 기본 LLM 테스트

목표

• Single-turn 테스트
• Multi-turn 테스트
• Embedding vector 테스트

코드

코드 링크: https://github.com/ybjeon/ai-agent-toyproject/blob/main/test_llm.py

• $ python test_llm.py

결과

Single-turn 테스트

=== Single Turn Test ===
>>>>>>> User: Hello! Introduce yourself briefly.
Assistant:  Hello there! I am a large language model trained by Mistral AI, designed to assist with a wide range of tasks and provide engaging conversation. I don't have personal experiences or emotions, but I'm here to help you navigate text-based tasks and make your interactions more enjoyable.

Multi-turn 테스트

=== Multi Turn Chat ===
Chatbot started (Server: http://localhost:11434) | Exit: 'quit' or 'exit'
>>>>>>> User: My name is Alice
Assistant:  Hello Alice, it's nice to meet you! How can I assist you today? Let me know if you have any questions or need help with something.
>>>>>>> User: What is my name?
Assistant:  Your name is Alice, according to the initial interaction we had. If you have a different name that you would like to use when interacting with me, just let me know and I'll update it for future conversations.

history에 채팅 기록을 계속 append 했기 때문에, 과거 입력이 다시 들어가서 Alice임을 안다.

Embedding vector 테스트

=== Embedding Vector Test ===
  The weather is really nice today: 0.4604
  Do you know about cats?: 0.6775
  Do you know about dogs?: 0.9181
  Dogs are animals with high loyalty: 0.6603
  Python is a programming language: 0.5940
  Cats have an independent personality: 0.4756
Query: Tell me about dogs
Closest: Do you know about dogs?

*중요 포인트: nomic-embed-text과 같이 embedding용 모델을 써야함