FIX: fix adaptation prompt CI and compatibility with latest transformers (4.35.0)

[younesbelkada]

What does this PR do?

As per title, fixes the current CI for adaption prompt and makes it compatible.
Transformers refactored the way positional embeddings are computed for llama, in the previous versions cos and sin cache values had a shape of length 4, now it was only a shape of length 2, breaking the assumptions we made in llama_apply_rotary_pos_emb.

I propose to keep the previous behaviour in case len(cos.shape) == 4, and fall back to the newest way of computing q_embed as depicted here: https://github.com/huggingface/transformers/blob/main/src/transformers/models/llama/modeling_llama.py#L222-L226

cc @BenjaminBossan @pacman100

[BenjaminBossan]

Thanks for identifying the issue. Could you please also undo these changes:

https://github.com/huggingface/peft/pull/1077/files#diff-2d7467165fbcbb3c9f83e0ca3ed12834e9e1a596176d01b10e8898e3f1058f55

[younesbelkada]

Good point Thanks! Done now!

[HuggingFaceDocBuilderDev]

The documentation is not available anymore as the PR was closed or merged.

[pacman100]

Thank you @younesbelkada for fixing this!

[BenjaminBossan]

Thanks, can be merged as soon as CI is green.