This item highlights common challenges encountered when working with local AI models, often stemming from the model’s harness, complexities in chat templates, and prompt construction. It also notes the presence of occasional inference bugs, indicating that issues can arise throughout the entire process from user input to result.
Source: Simon Willison