LLM 參數
與任何機器學習模型一樣，大型語言模型也有各種可控生成的文字 Output 變異性的參數。我們已經開始了一個多部分的系列，详盡解釋這些參數的影響。我們將通過 discusses 在這系列的各部分中平衡內容生成中使用所有這些參數。
歡迎來到第一部分，我們將討論最知名的參數 “Temperature。”
Temperature
如果目標是控制預測的隨機性，那麼 temperature 是您需要的。較低的 temperature 值會使輸出更具確定性，而較高的值將通過允許多樣化的結果來使其更具創造性。
讓我們通過以下代碼和輸出來看看 temperature 如何在实际行动中发挥作用。為了簡單地展示重要性，我們選擇使用 hugging face transformers 和 particular 的 GPT2 模型。
Python
33
1
import torch
2
from transformers import GPT2LMHeadModel, GPT2Tokenizer
3
4
# Load GPT-2 model and tokenizer
5
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
6
model = GPT2LMHeadModel.from_pretrained("gpt2")
7
8
# Add pad token to tokenizer (GPT-2 doesn't have it by default)
9
tokenizer.pad_token = tokenizer.eos_token
10
11
# Function to generate response with varying temperature
12
def generate_with_temperature(prompt, temperature):
13
inputs = tokenizer(prompt, return_tensors='pt', padding=True)
14
15
# Set the attention_mask and pad_token_id
16
outputs = model.generate(
17
inputs.input_ids,
18
attention_mask=inputs['attention_mask'],
19
do_sample=True,
20
max_length=200,
21
temperature=temperature,
22
pad_token_id=tokenizer.eos_token_id
23
)
24
return tokenizer.decode(outputs[0], skip_special_tokens=True)
25
26
#Prompt on which we get the content generated based on temperature
27
prompt = "What are some of the best strategies within NFL Fantasy"
28
29
# Test temperature variations
30
print("Temperature 0.3 (Less deterministic but coherent):\n", generate_with_temperature(prompt, temperature=0.3))
31
print("\nTemperature 0.5 (Balanced):\n", generate_with_temperature(prompt, temperature=0.5))
32
print("\nTemperature 0.9 (More creative):\n", generate_with_temperature(prompt, temperature=0.9))
33
Output:
PowerShell
45
1
python test_temperature.py
2
Temperature 0.3 (Less deterministic but coherent):
3
What are some of the best strategies within NFL Fantasy Football?
4
5
I've seen a lot of great strategies for predicting the next NFL season. I've seen great strategies for predicting the next NFL season.
6
7
I've seen a lot of great strategies for predicting the next NFL season. I've seen great strategies for predicting the next NFL season.
8
9
I've seen a lot of great strategies for predicting the next NFL season. I've seen great strategies for predicting the next NFL season.
10
11
I've seen a lot of great strategies for predicting the next NFL season. I've seen great strategies for predicting the next NFL season.
12
13
I've seen a lot of great strategies for predicting the next NFL season. I've seen great strategies for predicting the next NFL season.
14
15
I've seen a lot of great strategies for predicting the next NFL season. I've seen great strategies for predicting the next NFL season.
16
17
I've seen a lot of great strategies for predicting the next
18
19
Temperature 0.5 (Balanced):
20
What are some of the best strategies within NFL Fantasy Football?
21
22
1. Pick up the first pick and use it to draft a QB
23
24
In the past, I've mentioned that I tend to pick up the first pick in the first round. The best way to pick up the first pick is to take the first pick and then use it to draft a QB. I find that the more I draft the QB, the more I draft the QB. This is why I like to draft the first pick in the first round.
25
26
2. Draft a QB for the first time
27
28
This is an important one, but it's not the only one. Drafting a QB for the first time is a great way to start your fantasy season.
29
30
3. Draft a QB for the first time
31
32
This is the most important one. Drafting a QB for the first time is the most important one. Drafting a QB for the first time is the most important one.
33
34
Temperature 0.9 (More creative):
35
What are some of the best strategies within NFL Fantasy?
36
37
If you are looking for players that will be good for you, here is an updated list of key stats, which you can find on our official website:
38
39
All players were ranked in the top 10 fantasy players. These players are all high-rated defensive backs or running backs with good play across all phases of their careers. The players above were ranked from 5-5 for total points scored.
40
41
The chart below will allow you to visualize the players in your league.
42
43
All players have 5.5 sacks, 5 sacks and 2.5 tackles for loss on the season. They have a combined 11.3 sacks with a 4.6, 1.6 and 2.1 yards per carry average, respectively.
44
45
Each player has three touchdowns. The three touchdowns are tied for the top five fantasy points with 3 points in an entire game. The three touchdowns are tied for the top ten points with 2 points
讓我們來理解 Output:
- 低溫（0.3）:模型將集中在最可能詞選。如果精度與一致性對您來說很重要，那麼請給出 around this range 的 temperature。但是，請注意，模型可能會卡住重複類似的短語，正如我們的 Output 這裡一樣。
- 中溫（0.5）：這種溫度恰到好處地平衡了一致性和創造性。如果您想在保持結構的同時獲得一定程度的變異，這是一個非常好的中立點。正如您在輸出中所見，已經增加了一點平衡，但您仍然可以看到輸出中有些重複。
- 高溫（0.9）：這種溫度使得LLM（大型語言模型）尽可能具有創造性。正如您所見，這個輸出與前兩個不同，為內容帶來了大量的隨機性和變異。
上面的人才例子建立了一個基本的温度的理解。現在讓我們更詳細地看一下，用幾個使用案例：“創造性故事生成”和“技術解釋”。
讓我們用以下的代碼來了解溫度如何影響上述2個使用案例。
Python
x
1
import torch
2
from transformers import GPT2LMHeadModel, GPT2Tokenizer
3
4
# Load GPT-2 model and tokenizer
5
tokenizer = GPT2Tokenizer.from_pretrained("gpt2")
6
model = GPT2LMHeadModel.from_pretrained("gpt2")
7
8
# Add pad token to tokenizer (GPT-2 doesn't have it by default)
9
tokenizer.pad_token = tokenizer.eos_token
10
11
# Function to generate response based on temperature
12
def generate_with_temperature(prompt, temperature, max_length=200):
13
inputs = tokenizer(prompt, return_tensors='pt', padding=True)
14
outputs = model.generate(
15
inputs.input_ids,
16
attention_mask=inputs['attention_mask'],
17
do_sample=True,
18
max_length=max_length,
19
temperature=temperature, # Only focusing on temperature
20
pad_token_id=tokenizer.eos_token_id
21
)
22
return tokenizer.decode(outputs[0], skip_special_tokens=True)
23
24
### USE CASE 1: CREATIVE STORY GENERATION ###
25
26
def creative_story_generation():
27
prompt = "Once upon a time, in a distant galaxy, there was a spaceship called Voyager."
28
29
# Negative Impact: Low temperature for creative writing (too deterministic, repetitive)
30
print("\n=== Creative Story with Low Temperature (0.2) - Negative Impact: ===")
31
low_temp_story = generate_with_temperature(prompt, temperature=0.2)
32
print(low_temp_story)
33
34
# Perfect Impact: High temperature for creative writing (more creative and varied)
35
print("\n=== Creative Story with High Temperature (0.9) - Perfect Impact: ===")
36
high_temp_story = generate_with_temperature(prompt, temperature=0.9)
37
print(high_temp_story)
38
39
### USE CASE 2: TECHNICAL EXPLANATION ###
40
41
def technical_explanation():
42
prompt = "Explain how blockchain works in simple terms."
43
44
# Negative Impact: High temperature for technical writing (too creative, inaccurate)
45
print("\n=== Technical Explanation with High Temperature (0.9) - Negative Impact: ===")
46
high_temp_explanation = generate_with_temperature(prompt, temperature=0.9)
47
print(high_temp_explanation)
48
49
# Perfect Impact: Optimal temperature for technical writing (accurate and focused)
50
print("\n=== Technical Explanation with Adjusted Temperature (0.7) - Perfect Impact: ===")
51
perfect_temp_explanation = generate_with_temperature(prompt, temperature=0.7)
52
print(perfect_temp_explanation)
53
54
# Run both use cases
55
creative_story_generation()
56
technical_explanation()
57
輸出：
PowerShell
41
1
python temperature_impact.py
2
3
=== Creative Story with Low Temperature (0.2) - Negative Impact: ===
4
Once upon a time, in a distant galaxy, there was a spaceship called Voyager. It was a spaceship that had been brought back from the dead by the gods. It was a spaceship that had been brought back from the dead by the gods. It was a spaceship that had been brought back from the dead by the gods. It was a spaceship that had been brought back from the dead by the gods. It was a spaceship that had been brought back from the dead by the gods. It was a spaceship that had been brought back from the dead by the gods. It was a spaceship that had been brought back from the dead by the gods. It was a spaceship that had been brought back from the dead by the gods. It was a spaceship that had been brought back from the dead by the gods. It was a spaceship that had been brought back from the dead by the gods. It was a spaceship that had been brought back from the dead by the gods. It was a spaceship that had been
5
6
=== Creative Story with High Temperature (0.9) - Perfect Impact: ===
7
Once upon a time, in a distant galaxy, there was a spaceship called Voyager. The ship seemed to have been flying in space as well, like the spaceship in the movie "The Voyage Home". The Captain of the Voyager was Captain Jean-Luc Picard.
8
9
In the Star Trek movies, this is true. But in the movie, on Voyager, our hero is not Jean-Luc Picard. Jean-Luc, the villain, has no desire to lead us to vengeance against the dying star.
10
11
But what about Star Trek VI: The Undiscovered Country…
12
13
In a scene that's been in development for years (one you'll almost certainly see in an upcoming Star Trek film), in the middle of the movie (one you won't see until later in the film), we see that Picard is no longer in the Star Trek universe as is the story, to be played by the same actor who played Lieutenant Dax (who was in the movie), but Picard himself.
14
15
=== Technical Explanation with High Temperature (0.9) - Negative Impact: ===
16
Explain how blockchain works in simple terms.
17
18
Blockchain can be used to determine if the system is trustworthy and to prevent fraud, even if the system is used in a completely different manner. Blockchain can also be used to help determine how the system is run and to ensure that its operation is efficient.
19
20
This way all your trust in the system can be verified by your actions, and you can have full control over it. When you are not trusting a computer, it can be easy to get a hold of a server and then just change the software, allowing you to control and monitor transactions with the help of the blockchain. If your business uses distributed storage then it is easy to have more control over your activities.
21
22
What do I need to understand about blockchain?
23
24
To understand how blockchain works and how you can use it properly, you must first understand how blockchain works.
25
26
Bitcoins are digital tokens, created at the start of each generation that are used to
27
28
=== Technical Explanation with Adjusted Temperature (0.7) - Perfect Impact: ===
29
Explain how blockchain works in simple terms.
30
31
What are the key differences between Bitcoin and Ethereum?
32
33
Blockchain is a cryptographic protocol. It can be used to create any type of transaction. It is used to store data and create new entities. It is used as a system of communication in blockchain systems.
34
35
In Ethereum, the transaction is recorded, stored, and used to perform the transaction. It is a way to transfer information. The transaction is called a "blockchain."
36
37
Since the blockchain is used for many things, it is easy to understand how the technology works. The most important difference is that Ethereum uses the blockchain to create an interface to the Internet of Things. It is this interface that allows for data exchange and the creation of new entities.
38
39
Because of this, it is possible to perform the transactions on the blockchain. So, what is the difference between Bitcoin and Ethereum?
40
41
The Bitcoin and Ethereum blockchain is a distributed ledger.
現在讓我們停下来分析根據溫度設定和輸出受影響情況下的創造性故事生成和技術解釋的輸出。我們還將觀察一個使用案例中溫度設定如何恰到好處，而另一個使用案例中則恰恰相反。
創造性故事生成
- 低溫（負面影響）：正如您所見，故事輸出高度重複且缺乏變體。對於創造性任務來說，這種結果是不滿意的，模型無法引入新穎和创新的想法導致極端的重複性，使其不適合故事叙述。
- 高溫（完美影響）從 Output 可以看出，故事采取了有趣的方向並且非常具有創造性。 Output 還為故事添加了多方面的內容，使其變得豐富多樣、富有想像力，並且非常适合創新 storytelling。
技術說明
- 高溫（負面影響）重要的是要牢记，對於像技術說明這樣的用例，保持事實準確性非常重要的是。高溫導致生成的內容中引入了大量的隨機性和不太可能出現的詞語，這使其對於技術寫作來說不滿意。從上面的 Output 也可以推断出，它太模糊並且包含不相關的思想。
- 調整溫度（完美影響）我們已將溫度調整到一種能夠為生成技術內容 Strikes a perfect balance 的設定。如您所见， Output 現在 classification 很多。在這個溫度設定下，模型避开了像在較低溫度下一樣的repetitiveness 並且不失coherence 像在較高溫度下一樣。
結論
您已經看到了各種方式会影响内容生成以及哪种温度设置适用于哪种用例。另外，请注意，调整温度并不是生成内容的全部；您还需要调整其他参数。我们将在本系列接下来的文章中介绍所有这些内容。
Source:
https://dzone.com/articles/decoding-llm-parameters-temperature