Flash attention support

by jiosephlee - opened Oct 25

Oct 25

Hi, I seem to be getting different performance based on the attention implementation. Is flash attention 2 also supported? Or just flash attention 1.

Intern Large Models org Oct 27

•

InternS1 follow the attention interface introduced in huggingface：https://huggingface.co/docs/transformers/attention_interface，you can set attn_implementation to control the attention implementation.

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment