Reinforces two of my strong suspicions:
1 - Small fine-tunes are *way* under-explored.
2 - Getting LLMs to speak JSON is increasing an architecture smell. In almost every situation you are getting degraded performance vs. asking it to speak XML, YAML, etc.
add a skeleton here at some point
about 1 month ago