Exploring the Latest Oobabooga with Guanaco Instructions and Chat Settings
The latest oobabooga model has shown significant advancements, particularly with the incorporation of the guanaco instruction template in the Chat Settings and the "Chat-Instruct" mode.
The model handles long, detailed prompts exceptionally well, maintaining coherence even for a 2000 token count. Moreover, it successfully identifies potential multiple interpretations and outcomes of its responses. This feature can be beneficial, as it provides balanced, thoughtful, and informative responses instead of generic or overly cautious statements.
One question that arises is about the commercial usability of these models, as they are built on top of llama, which is non-commercial. The exploration of Openllama models with guanaco based tuning presents an interesting prospect.
Regarding model choices, based on the elo evaluation by GPT4, Vicuna-13B outperforms Guanaco-13B. It suggests that, for those of us with hardware constraints, Vicuna or Vicuna-based models are more suitable.
Users have also explored the possibility of converting a 4-bit GGML model back into a PyTorch model, retaining the 4-bit quantization, or converting a 4-bit GPTQ .safetensors model into a CoreML model. These are interesting avenues to explore and require more research.
Users have reported the Guanaco 33B model to perform well in role-play scenarios and generating smart, meaningful, and uncensored content. However, it might fill up the context quickly, which may require asking the model for more concise replies.
As always, there are ongoing discussions and debates on the choice between GGML and GPTQ. Different users might prefer one over the other based on their specific needs and experiences. While the GGML model might break Windows compatibility unless used with WSL2, GPTQ is more Windows-friendly.
Lastly, an appreciation for the effort put in by individuals, like u/The-Bloke, to quantize these models and make them readily available for use, cannot be understated.
Tags: oobabooga, guanaco, llama, models, GPT4, ChatInstruct, Openllama, Vicuna, GGML, GPTQ