LLM參數
像任何機器學習模型一樣，大型語言模型具有控制生成文本輸出差異的各種參數。我們已經開始了一個多部分系列，以詳細解釋這些參數的影響。我們將通過在我們的多部分系列中討論的所有這些參數來達到內容生成的完美平衡。
歡迎來到第二部分，在這裡我們將討論另一個眾所周知的參數，“Top-P”。
Top-P（核子取樣）
如果目標是控制模型輸出的多樣性，那麼Top-P就是您的不二之選。較低的Top-P強迫模型使用最可能的單詞，而較高的Top-P則迫使模型使用更多樣化的單詞，增加創造力。
讓我們通過以下代碼和輸出來看看Top-P的作用。
Python
import torch
from transformers import GPT2LMHeadModel, GPT2Tokenizer
# Load GPT-2 model and tokenizer
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
model = GPT2LMHeadModel.from_pretrained("gpt2")
# Add pad token to tokenizer (GPT-2 doesn't have it by default)
tokenizer.pad_token = tokenizer.eos_token
# Function to generate response with varying top_p
def generate_with_top_p(prompt, top_p):
inputs = tokenizer(prompt, return_tensors='pt', padding=True)
# Set the attention_mask and pad_token_id
outputs = model.generate(
inputs.input_ids,
attention_mask=inputs['attention_mask'],
do_sample=True,
max_length=200,
top_p=top_p,
pad_token_id=tokenizer.eos_token_id
)
return tokenizer.decode(outputs[0], skip_special_tokens=True)
prompt = "What are some effective ways to manage stress in daily life?"
# List of top-p values and their descriptions
top_p_values = {
0.1: "Very conservative: Generates highly probable and safe responses.",
0.3: "Conservative: Generates probable responses with less risk.",
0.5: "Balanced: A mix of safe and creative responses.",
0.7: "Creative: Generates more diverse and creative responses.",
0.9: "Very creative: Allows for highly diverse and less probable responses."
}
# Test top_p variations
for top_p, description in top_p_values.items():
print(f"\nTop-p {top_p} ({description}):\n")
print(generate_with_top_p(prompt, top_p=top_p))
輸出：
PowerShell
python test_top_p.py
Top-p 0.1 (Very conservative: Generates highly probable and safe responses.):
What are some effective ways to manage stress in daily life?
The following are some of the most common ways to manage stress in daily life.
1. Avoiding stress
The most common way to avoid stress is to avoid it.
The most common way to avoid stress is to avoid it.
2. Avoiding stress
The most common way to avoid stress is to avoid it.
The most common way to avoid stress is to avoid it.
3. Avoiding stress
The most common way to avoid stress is to avoid it.
The most common way to avoid stress is to avoid it.
4. Avoiding stress
The most common way to avoid stress is to avoid it.
The most common way to avoid stress is to avoid it.
5. Avoiding stress
The most common way to avoid stress is to avoid it.
The most common way to avoid stress
Top-p 0.3 (Conservative: Generates probable responses with less risk.):
What are some effective ways to manage stress in daily life?
What are some effective ways to manage stress in daily life?
What are some effective ways to manage stress in daily life?
What are some effective ways to manage stress in daily life?
What are some effective ways to manage stress in daily life?
What are some effective ways to manage stress in daily life?
What are some effective ways to manage stress in daily life?
What are some effective ways to manage stress in daily life?
What are some effective ways to manage stress in daily life?
What are some effective ways to manage stress in daily life?
What are some effective ways to manage stress in daily life?
63
What are some effective ways to manage stress in daily life?
64
65
What are some effective ways to manage stress in daily life?
66
67
What are some effective ways to manage stress in daily life?
68
69
What are some effective
70
71
Top-p 0.5 (Balanced: A mix of safe and creative responses.):
72
73
What are some effective ways to manage stress in daily life?
74
75
1. Stay on top of your body's natural stress levels
76
77
When you're stressed, your body's natural stress levels are low.
78
79
If you're stressed, your body's natural stress levels are high.
80
81
If you're stressed, your body's natural stress levels are low.
82
83
2. Avoid excessive exercise
84
85
Exercise can make you feel better.
86
87
Exercise can make you feel better.
88
89
3. Get up early to avoid fatigue
90
91
Exercise can make you feel better.
92
93
Exercise can make you feel better.
94
95
4. Avoid the temptation to take the wrong thing
96
97
Exercise can make you feel better.
98
99
Exercise can make you feel better.
100
101
5. Avoid eating the wrong foods
102
103
Exercise can make you feel better.
104
105
Exercise can make you feel better.
106
107
6. Avoid the temptation to
108
109
Top-p 0.7 (Creative: Generates more diverse and creative responses.):
110
111
What are some effective ways to manage stress in daily life?
112
113
I am talking about a very specific situation. The person I am talking about has been stressed, but has not been doing much work for a long time. I want to tell you, because this person has had a lot of stress in his life, that it is not something you can just go back to. But what I'm trying to say is, that if you don't have a job, you have to go back to work every day, so you can spend more time with your family. So I've been doing that for a long time now. And so, that is a very common occurrence.
114
115
But what do you think is the best way to deal with the stress?
116
117
You know, it's not easy to deal with it. It is very difficult to deal with the stress that we experience. So, that is a very good way to deal with it. So, I think it's the
118
119
Top-p 0.9 (Very creative: Allows for highly diverse and less probable responses.):
120
121
What are some effective ways to manage stress in daily life?
122
123
There are many things that can be done by daily meditation and practice. As a general rule of thumb, meditation can help you stay mindful of your own needs, goals, feelings, desires, emotions, and the life and emotions around you.
124
125
The purpose of meditation is to feel a deep desire to practice more, to be more mindful, and to be more productive. It also serves to enhance your overall well-being.
126
127
1. Be active, be creative, be mindful, and be optimistic.
128
129
This is where the first step towards meditation comes from. If we're looking for inspiration, there's a whole section on being "active" and "creative."
130
131
While I'm not sure I know much about meditation, I know some of its practitioners and some that I never met. My mom used to tell me that she'd always find a way to make her feel more connected and involved.
現在讓我們了解輸出。
- Top-P 0.1 – 非常保守：由於模型從概率後10%的選擇中進行選擇，生成的內容中有很多重複。因此，這個回應缺乏多樣性，大部分時間也不具信息性。
- 顶端P 0.3 – 保守:模型從可能性較高的詞語選擇中選取前30%，因此它比上一个頂端P設定稍微不那麼保守。從输出来看，這并没有改進內容生成，而且补全中不斷重複提示。在這種情況下，提示的重複意味著模型認為在提示後最有可能的繼續是提示本身。
- 顶端P 0.5 – 平衡:在这里，您可以看到模型第一次列出了一些编号策略。在此設定中，您仍然可以看到一些重複。但归根結底，在這個頂端P設定下，模型開始纳入更廣泛的詞語。輸出是標準建議與一些不一致性的混合。這個頂端P值允許提高創造性，但仍然 struggling 著信息的深度。
- 顶端P 0.7 – 創造性:在這種情況下，模型可以從更廣泛的詞語中選擇，正如您所見，響應正在向叙事情節風格轉變。內容更具有創造性，因為現在它涉及一個人如何應對压力的情境。不便的是失去了焦点，因為強調的是應對压力的困難，而不是管理压力。
- 顶端P 0.9 – 非常創造性:在此設定中，模型可以訪問廣泛的詞彙和想法，包括可能性較低的詞語和概念。此設定使得模型能夠使用更表達性的語言。同樣，非常創造性的不便之处是，模型在生產豐富多變的內容的過程中偏离了提示。
上述練習的重要觀察點是，內容如何隨著Top-P設定值的變化而變化。它也讓我們了解到，這個參數並不是唯一需要處理變化和相關性的 parameter。
現在，讓我們看看Top-P對幾個使用案例的影響，就像本系列的 “
Python
1
import torch
2
from transformers import GPT2LMHeadModel, GPT2Tokenizer
3
4
# Load GPT-2 model and tokenizer
5
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
6
model = GPT2LMHeadModel.from_pretrained("gpt2")
7
8
# Add pad token to tokenizer (GPT-2 doesn't have it by default)
9
tokenizer.pad_token = tokenizer.eos_token
10
11
# Function to generate response based on top_p
12
def generate_with_top_p(prompt, top_p, max_length=250):
13
inputs = tokenizer(prompt, return_tensors='pt')
14
outputs = model.generate(
15
inputs.input_ids,
16
attention_mask=inputs.attention_mask,
17
do_sample=True,
18
max_length=max_length,
19
top_p=top_p,
20
pad_token_id=tokenizer.eos_token_id,
21
eos_token_id=tokenizer.eos_token_id,
22
no_repeat_ngram_size=2 # Prevents repetition of phrases
23
)
24
return tokenizer.decode(outputs[0], skip_special_tokens=True)
25
26
### USE CASE 1: CREATIVE STORY GENERATION ###
27
28
def creative_story_generation():
29
prompt = ("In the mystical land of Eldoria, a young warrior found an ancient map "
30
"that led to a hidden treasure guarded by dragons. He knew that courage and "
31
"wisdom would be his allies on this perilous journey.")
32
33
# Negative Impact: Low top_p for creative writing (less creative)
34
print("\n=== Creative Story with Low top_p (0.2) - Negative Impact: ===")
35
low_top_p_story = generate_with_top_p(prompt, top_p=0.2)
36
print(low_top_p_story)
37
38
# Perfect Impact: High top_p for creative writing (more creative)
39
print("\n=== Creative Story with High top_p (0.95) - Perfect Impact: ===")
40
high_top_p_story = generate_with_top_p(prompt, top_p=0.95)
41
print(high_top_p_story)
42
43
### USE CASE 2: TECHNICAL EXPLANATION ###
44
45
def technical_explanation():
46
prompt = ("Explain step by step how the internet works, focusing on how computers "
47
"use IP addresses and data packets to communicate with each other.")
48
49
# Negative Impact: High top_p for technical writing (less precise)
50
print("\n=== Technical Explanation with High top_p (0.95) - Negative Impact: ===")
51
high_top_p_explanation = generate_with_top_p(prompt, top_p=0.95)
52
print(high_top_p_explanation)
53
54
# Perfect Impact: Optimal top_p for technical writing (accurate)
55
print("\n=== Technical Explanation with Optimal top_p (0.5) - Perfect Impact: ===")
56
optimal_top_p_explanation = generate_with_top_p(prompt, top_p=0.5)
57
print(optimal_top_p_explanation)
58
59
# Run both use cases
60
creative_story_generation()
61
technical_explanation()
62
創意故事生成的 ‘
PowerShell
1
python top_p_multiple.py
2
3
=== Creative Story with Low top_p (0.2) - Negative Impact: ===
4
In the mystical land of Eldoria, a young warrior found an ancient map that led to a hidden treasure guarded by dragons. He knew that courage and wisdom would be his allies on this perilous journey.
5
6
The Dragon King
7
...
8
(The Book of the Dragon)
9
,
10
-
11
: The Dragon Lord is a legendary warrior who has been the focus of many legends. The dragon king is the most powerful of all the dragons in the world. In the magical land, he is known as the "Dragon King". He is also known to be the leader of a group of dragons called the Black Dragons. His name is derived from the dragon's name, "the dragon".
12
"The Black Dragon" is an important symbol of power and powerlessness. It is said that the black dragon is able to create a dragon that can defeat the strongest of his enemies. However, the true power of this dragon lies in his ability to manipulate the minds of others. This ability is called "The Dark Dragon". The Dark dragon has a powerful sense of self-preservation and is capable of manipulating others to his will. When he has control over others, his power is so great that he can destroy entire cities. As a result
13
14
=== Creative Story with High top_p (0.95) - Perfect Impact: ===
15
In the mystical land of Eldoria, a young warrior found an ancient map that led to a hidden treasure guarded by dragons. He knew that courage and wisdom would be his allies on this perilous journey.
16
17
Spirits are like gods. In this world, there are no gods without secrets. There are also no secrets about being a fighter or a thief. But every dragon has a special hidden skill, and he or she can use that skill to destroy and gain strength or hide something hidden in the secret. Many dragons are skilled at their martial arts, while most are unaware of the secrets of their true power. These dragons cannot only use these skills, but that will only allow them to escape the dragons' clutches. Because their training will be tested before they're even born, dragon fighting has never been so hard, even without training, so they should be able to break a dragon's body.
18
19
=== Technical Explanation with High top_p (0.95) - Negative Impact: ===
20
Explain step by step how the internet works, focusing on how computers use IP addresses and data packets to communicate with each other. If a person with the same identity as a user on the US government's private network uses the online address bar, then this data is sent to a server on a computer on your local network. Your IP address is a small byte in the string. The IP and network address are identical. Do you remember, you just want to do that instead of using IPs or numbers. In addition, remember that IP can be used to verify a particular IP for you and your computer. For instance, your name does not always match an address on our government network and you should have your public IP in this country. This does seem quite unusual and perhaps a bit bizarre.
21
22
There was a time in Silicon Valley when you could set your identity out. But in most of today's world, how do you set up your own address and how does one look for it? What about the public? The internet itself was different. It was just a set of rules around data flow that you were supposed to follow. Now, even in today the "internet in general" seems a little more complicated to define. Let's say
23
24
=== Technical Explanation with Optimal top_p (0.5) - Perfect Impact: ===
25
Explain step by step how the internet works, focusing on how computers use IP addresses and data packets to communicate with each other.
26
27
"We've been trying to understand how it works and what it means for the future," says James. "It's not just about the IP address, it's about how people communicate. It's also about what's going on with the data. We want to see how this works. What is the Internet going to look like in the next 10 years?"
28
, the director of the Computer Science and Artificial Intelligence Laboratory at the University of Michigan, says that while there's still a lot of work to be done, "we've got to start to think about it."
一部分和”技術解釋” ‘
。 Output:
現在讓我們分解並分析基于 Top-P 設定值和 Output 影响的創意故事生成和技術解釋 Output。
- 為了有效地展示 Top-P 的影響，我們已將 prompt 更好地理性地引导 Output，以便輕鬆观察影響。創意故事生成
- 低 Top-P (負面影響):如您所見，低 Top-P 下，模型受到单词或短語的使用限制，因此導致重複和冗餘。在這種情況下，創造性也受到限制，因為模型嘗試不引入新想法。但如果您注意到了， logical flow 仍然得到維護，並且 model 保持在主題上，這通常是低 Top-P 值的特點。
技術解釋
- 高Top-P（負面影響）：如你所見，高Top-P對技術解釋產生負面影響，因為它會阻止邏輯流的進行並使內容偏离主題。模型的這種作為亦引入了與解釋無關的信息。
- 最優Top-P（完美影響）：在最優Top-P的情況下，解釋的 条理更加清晰，且貼近主題。內容與提示更加對應，且在準確性和表達之間取得了良好的平衡。由於模型生成的文字受到更有機概率詞的限制，因此提高資訊的可靠性。
結論
通過這次實驗，我們成功地展示瞭Top-P參數在控制生成文字的隨機性和創造性方面的重要性。我們首先观察到一个提示下，隨著Top-P變化的輸出如何變化，然後采用更具應用案例方法的觀察，了解在不同用例下Top-P如何控制輸出。
然而，從本系列前文和本文的观察中，我們發現單獨每個參數對於生成內容的質量來說還是不夠公正。那就是為什麼研究所有這些參數的影響至關重要，我們将在本系列的最後一部分對其進行探讨。
Source:
https://dzone.com/articles/decoding-llm-parameters-top-p