Chapter 56
2. 为什么要微调 Why finetune
Notebookgpt24 cells
2. 为什么要微调 Why finetune
在这节课中，我们将学习为什么要进行微调，微调到底是什么。将其与提示词工程进行比较，并通过实验比较一个经过微调的模型和一个非微调模型。
那么，为什么要对大模型进行微调呢?在我们讨论原因之前，让我们来认识一下“什么是微调”。微调就是把这些通用的东西像GPT-3这样的模型，并对它们进行专业化变成ChatGPT之类的东西，具体的聊天用例，使它能很好地聊天，或者使用GPT-4，将它变成一个专门的GitHub代码助手，自动完成补全代码。
目录
- 2.1 尝试一个没有经过微调的模型
- 2.2 比较一下经过微调的模型
- 2.3 与chatPT作比较
In [1]python · cell 4
python
from llama import BasicModelRunner2.1 尝试一个没有经过微调的模型
In [2]python · cell 6
python
non_finetuned = BasicModelRunner("meta-llama/Llama-2-7b-hf", config={"production.key": "<YOUR-KEY-HERE>"})In [3]python · cell 7
python
print(non_finetuned("Tell me how to train my dog to sit"))Output
. Tell me how to train my dog to stay. Tell me how to teach my dog to come. Tell me how to get my dog to heel. Tell me how to stop my dog from jumping. Tell me how to make my dog stop barking. Tell me how I can get my dog to stop chewing. Tell me how to house train my dog. Tell me how to potty train my dog. Tell my how to train my dog to walk on a leash. Tell me how to crate train my dog. Tell Me How To Train My Dog To Sit, Stay, Come, Heel, Stop Jumping, Stop Barking, Stop Chewing, House Train, Potty Train, Walk On A Leash, Crate Train, And More! Tell Me How To Train Your Dog To Sit, Stay, Heel, Come, Stop Jumping, Stop Chewing, House Trained, Potty Trained, Walk On A Leash, And More! Tell me how to train your dog to sit. Tell Me How To House Train My Dog.
In [4]python · cell 8
python
print(non_finetuned("What do you think of Mars?"))Output
I think it's a great planet. I think it's a good planet. I think it'll be a great planet. I think we should go there. I think we should go back there. I think we should stay there. I think we should leave there. I think we should colonize there. I think we should terraform there. I think we should mine there. I think we should build there. I think we should live there. I think we should die there. I think we should be there. I think we should have been there. I think we should never be there. I think we'll be there. I think we won't be there. I think we will be there. I think we can be there. I think we could be there. I think we would be there. I think we might be there. I think we may be there. I think we shall be there. I think we must be there. I think we have to be there. I think we need to be there. I think I'll be there. I know I'll
In [5]python · cell 9
python
print(non_finetuned("taylor swift's best friend"))Output
I'm not sure if I've mentioned this before, but I'm a huge Taylor Swift fan. I've been a fan since her first album, and I've been a fan ever since. I've been a fan of her music, her style, her personality, and her music. I've been a fan for a long time. I’ve been a fan of Taylor Swift for a long time. I’ve been a fan of her music for a long time. I’m a fan of her style, her personality, her music, and her music. I’ve been a fan for a long, long time. I’ve always been a fan of Taylor Swift. I’ve always been a fan of her music. I’ve always been a big fan of her style. I’ve always been a huge fan of her personality. I’ve always been a massive fan of her music. I’m a fan of Taylor Swift. I love her music. I love her style. I love her personality. I love her music. I’m a huge fan of Taylor Swift. I’ve always been an avid fan of Taylor Swift. I’m
In [6]python · cell 10
python
print(non_finetuned("""Agent: I'm here to help you with your Amazon deliver order.
Customer: I didn't get my item
Agent: I'm sorry to hear that. Which item was it?
Customer: the blanket
Agent:"""))Output
I'm sorry to hear that but I'm here to help. Customer: I don' t get the blanket Customer: I don't have the blanket Agent: I don't understand. Agent: I'm sorry. I don't understand. customer: I don't have the blanket Agent: I'm so sorry to hear that. Customer: I don t have the blanket Agent: Oh no, I'm so sorry to hear about that. customer: I
注：从以上的示例中可以看到，没有经过微调的大模型在面对一些很简单的问题的时候，也会出现胡言乱语的情况。
2.2 比较一下经过微调的模型
In [8]python · cell 13
python
finetuned_model = BasicModelRunner("meta-llama/Llama-2-7b-chat-hf",config={"production.key": "<YOUR-KEY-HERE>"})In [9]python · cell 14
python
print(finetuned_model("Tell me how to train my dog to sit"))Output
on command. How to Train Your Dog to Sit on Command Training your dog to sit on command is a basic obedience command that can be achieved with patience, consistency, and positive reinforcement. Here's a step-by-step guide on how to train your dog to sit on command: 1. Choose a Quiet and Distraction-Free Area: Find a quiet area with minimal distractions where your dog can focus on you. 2. Have Treats Ready: Choose your dog's favorite treats and have them ready to use as rewards. 3. Stand in Front of Your Dog: Stand in front of your dog and hold a treat close to their nose. 4. Move the Treat Above Your Dog's Head: Slowly move the treat above your dog's head, towards their tail. As your dog follows the treat with their nose, their bottom will naturally lower into a sitting position. 5. Say "Sit" and Reward: As soon as your dog's butt touches the ground, say "Sit" and give them the treat. It's important to say the command word as they're performing
In [10]python · cell 15
python
print(finetuned_model("[INST]Tell me how to train my dog to sit[/INST]"))Output
Training your dog to sit is a basic obedience command that can be achieved with patience, consistency, and positive reinforcement. Here's a step-by-step guide on how to train your dog to sit: 1. Choose a quiet and distraction-free area: Find a quiet area with no distractions where your dog can focus on you. 2. Have treats ready: Choose your dog's favorite treats and have them ready to use as rewards. 3. Stand in front of your dog: Stand in front of your dog and hold a treat close to their nose. 4. Move the treat up and back: Slowly move the treat up and back, towards your dog's tail, while saying "sit" in a calm and clear voice. 5. Dog will sit: As you move the treat, your dog will naturally sit down to follow the treat. The moment their bottom touches the ground, say "good sit" and give them the treat. 6. Repeat the process: Repeat steps 3-5 several times, so your dog starts to associate the command "sit" with the action of sitting down. 7.
In [11]python · cell 16
python
print(non_finetuned("[INST]Tell me how to train my dog to sit[/INST]"))Output
[INST]Tell me how to teach my dog to sit[/INST][/INST] [INST]How to teach my dog to sit[INST] [INST]How do I teach my dog to sit[INST][/INST] [inst]How to teach my dog to stay[/inst] [inst]How to train my dog to stay[/inst][/inst] [inst]Tell me how to train a dog to stay[/inst]
In [12]python · cell 17
python
print(finetuned_model("What do you think of Mars?"))Output
Mars is a fascinating planet that has captured the imagination of humans for centuries. It is the fourth planet from the Sun in our solar system and is known for its reddish appearance. Mars is a rocky planet with a thin atmosphere, and its surface is characterized by volcanoes, canyons, and impact craters. One of the most intriguing aspects of Mars is its potential for supporting life. While there is currently no evidence of life on Mars, the planet's atmosphere and geology suggest that it may have been habitable in the past. NASA's Curiosity rover has been exploring Mars since 2012, and has discovered evidence of water on the planet, which is a key ingredient for life. Mars is also a popular target for space missions and future human settlements. Several space agencies and private companies are planning missions to Mars in the coming years, with the goal of establishing a human presence on the planet. The challenges of establishing a human settlement on Mars are significant, including the harsh environment, lack of resources, and distance from Earth. However, many experts believe that Mars is the next logical step in the expl
In [13]python · cell 18
python
print(finetuned_model("taylor swift's best friend"))Output
Taylor Swift's best friend is a person who has been by her side through thick and thin. Here are some possible candidates: 1. Abigail Anderson - Abigail is Taylor's childhood friend and has been a constant presence in her life. The two have been inseparable since they met in kindergarten and have shared countless memories together. 2. Selena Gomez - Selena and Taylor have been friends for over a decade and have been through a lot together. They have collaborated on music projects, gone on vacation together, and have been there for each other through personal struggles. 3. Liz Rose - Liz is a songwriter and producer who has worked with Taylor on many of her hit songs. The two have a close creative partnership and have been friends for many years. 4. Joe Jonas - Joe and Taylor were friends for several years before they started dating in 2008. They have been on-again, off-again since then and have been there for each other during some of the most significant moments in each other's lives. 5. Ed Sheeran - Ed and Taylor have been friends
In [14]python · cell 19
python
print(finetuned_model("""Agent: I'm here to help you with your Amazon deliver order.
Customer: I didn't get my item
Agent: I'm sorry to hear that. Which item was it?
Customer: the blanket
Agent:"""))Output
I see. Can you please provide me with your order number so I can look into this for you? Customer: I don't have the order number. Agent: Okay, no worries. Can you please tell me the name of the item you didn't receive? Customer: Blanket Agent: And what was the shipping address for this item? Customer: 123 Main St Agent: Okay, thank you. I'm going to check on the status of your order. Can you please hold for just a moment? Customer: (hangs up) This is a common scenario in customer service, where the agent is trying to gather information from the customer to help resolve their issue. The agent is using active listening skills to gather information from the customer, such as the order number, the name of the item, and the shipping address. The agent is also using empathy to acknowledge the customer's concern and let them know that they are there to help.
注：经过微调的大模型，回答的就比较好，最起码是回答了问题的内容。
2.3 与chatPT作比较
In [15]python · cell 22
python
chatgpt = BasicModelRunner("chat-gpt", config={"production.key": "<YOUR-KEY-HERE>"})In [16]python · cell 23
python
print(chatgpt("Tell me how to train my dog to sit"))Output
Training a dog to sit is a basic command that can be taught using positive reinforcement techniques. Here's a step-by-step guide on how to train your dog to sit: 1. Choose a quiet and distraction-free environment: Find a calm area in your home or a quiet outdoor space where your dog can focus on the training without any distractions. 2. Gather treats: Use small, soft, and tasty treats that your dog loves. These treats will serve as rewards during the training process. 3. Get your dog's attention: Call your dog's name or use a clicker to get their attention. Make sure they are looking at you before proceeding. 4. Lure your dog into a sitting position: Hold a treat close to your dog's nose and slowly move it upwards and slightly backward over their head. As their nose follows the treat, their bottom will naturally lower into a sitting position. Once they are sitting, say "sit" in a clear and firm voice. 5. Reward and praise: As soon as your dog sits, give them the treat and offer verbal praise such as "good sit" or "well done." This positive reinforcement will help them associate the action with the command. 6. Repeat the process: Practice the sit command multiple times in short training sessions
注：当然chatgpt可以做到更好。