What is the prompt template of this model?
#1
by Shrelic - opened
Does this model have a prompt template?
ReluLLaMA models have not undergone Supervised Fine-Tuning (SFT), hence there isn't a designated prompt template for conversational tasks. However, utilizing general-purpose templates such as "Question: ...\nAnswer: ..." may prove beneficial for guiding the model's responses.
Hello, I'd like to know whether RelluLlama, like ReluStrikeback, also adds two more activation functions (after normalization) while replacing the activation function in MLP with ReLU. Or rather, does each layer of RelluLlama have three ReLU functions or just one ReLU function?