The code is similar to the GPT example in this repo:

from transformers import AutoTokenizer, AutoModel
from bertviz import head_view
from bertviz import model_view

# load the model
# Vicuna is an instruction-model based on Llama
model_name = "lmsys/vicuna-7b-delta-v1.1" # mistralai/Mistral-7B-Instruct-v0.1
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModel.from_pretrained(model_name, output_attentions=True)

input_sentence = """The sentence you are given might be too wordy, complicated, or unclear. Rewrite the sentence and make your writing clearer by keeping it concise. Whenever possible, break complex sentences into multiple sentences and eliminate unnecessary words.\n
Input: If you have any questions about my rate or if you find it necessary to increase or decrease the scope for this project, please let me know.\n\nOutput:"""
input_sentence = "Generate a positive review for a place."
inputs = tokenizer.encode(input_sentence, return_tensors='pt')
outputs = model(inputs)
attention = outputs[-1]  # Output includes attention weights when output_attentions=True
tokens = tokenizer.convert_ids_to_tokens(inputs[0]) 

# save the complete model view
html_head_view = head_view(attention, tokens, html_action='return')
with open("all_head_view.html", 'w') as file:
    file.write(html_head_view.data)

html_model_view = model_view(attention, tokens, html_action='return')
with open("all_model_view.html", 'w') as file:
    file.write(html_model_view.data)

# save the view just for certain layers if the browser can't display the whole
# shorter inputs are easier to display
layers = [1]
html_head_view = head_view(attention, tokens, html_action='return', include_layers=layers)

with open("short_head_view.html", 'w') as file:
    file.write(html_head_view.data)

html_model_view = model_view(attention, tokens, html_action='return', include_layers=layers)
with open("short_model_view.html", 'w') as file:
    file.write(html_model_view.data)

The loading and processing already take 30 GB of RAM. My machine starts to swap at this point and i just save the html to visualize it after the RAM is free again.

The output looks very repetitive.

In the case of Vicuna (lmsys/vicuna-7b-delta-v1.1 From Language Modeling to Instruction Following: Understanding the Behavior Shift in LLMs after Instruction Tuning) all heads consist of the same weight shape. Every token can only attend to its previous tokens due to the one-directional objective of GPTS, e.g. the first token can only attend to the start of the sentence token and itself. Interestingly, every token equally shares its weights to all possible tokens. Therefore, the attention weights are strongest for the tokens at the beginning and decrease towards the end. This entails a "L" shape like the positive multiplicative inverse.

Let me know if you find other patterns or have a good explanation for this phenomenon

Any plan on upadating the code for LLaMA models? #128

Description

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions