Fix: avoid passing None decoder_attention_mask to generation_kwargs

#35

by vamsin07 - opened about 16 hours ago

base: refs/heads/main

←

from: refs/pr/35

Discussion Files changed

-1

vamsin07

about 16 hours ago

Fixes the decoder_attention_mask=None crash in model.generate() on transformers>=4.55. Details + reproducer in discussion #34. One-line guard so we don't insert None into model_kwargs.

Fix: avoid passing None decoder_attention_mask to generation_kwargs8fb975b8

julianmack

Cohere Labs org about 10 hours ago

Thanks for this but we haven't tested this generate( + trust_remote_code=True code path. We will continue to fix bugs in the trust_remote_code=True path that are surfaced when running according to the original docs but we don't intend to widen the API surface for the deprecated version as it has the chance to break people's existing workflows using that code path.

See here for context: https://huggingface.co/CohereLabs/cohere-transcribe-03-2026/discussions/34#69e5f33b73519ae4cc18199a

Thanks again!

julianmack changed pull request status to closed about 10 hours ago

Upload images, audio, and videos by dragging in the text input, pasting, or clicking here.

Tap or paste here to upload images

· Sign up or log in to comment