Hugging Face's logo

bigscience
/
bloomz

Text Generation
Transformers
PyTorch
TensorBoard
Safetensors
bloom
Eval Results (legacy)
text-generation-inference
Model card Files
xet
 Metrics Community
56

Mt0 or Bloomz model: results are always same short length?

#56
by dsbyprateekg - opened
Discussion
dsbyprateekg

I tried to run the official prompt with or without max_new_tokens with bigscience/mt0-base model, but got the same length response:

image.png

image.png

With Bloomz model, I am not seeing even a output-

image.png

Can you please tell me what I am missing here?
Muennighoff
 BigScience Workshop org

For BLOOMZ, try set min_new_tokens

For mT0 maybe the new tokens are all special tokens that are removed or so - can you check the shape of the output? I.e. does it correspond to >= 2000
dsbyprateekg

@Muennighoff for MT0, shape is torch.Size([1, 20])

image.png

And for BLOOMZ, it gave the unexpected output-

image.png
Muennighoff
 BigScience Workshop org

for mt0, it tells you in the warning that 2000 is too big

for bloomz the output looks ok to me. you may get better perf doing e.g. {text}. Please translate the prior text to English.
dsbyprateekg

@Muennighoff for mt0, I updated and now the output is like below-

image.png

And for bloomz, output is now-

image.png
Muennighoff
 BigScience Workshop org

for {text} i meant that you should put your text 😂
dsbyprateekg

my bad, I updated-

image.png

Sign up or log in to comment