Technical Context
I love comparisons like this not for the charts, but for how grounded they are. This isn't about synthetic data or a model solving an Olympiad problem; it's about 69 real call center transcripts run through the exact same prompt. For AI automation, this is less theoretical and more like a nearly complete piece of a support pipeline.
The source here isn't an official vendor release but a practical test from the community. That's why I see it as a field benchmark rather than the absolute truth. But these kinds of tests are usually more useful than marketing PDFs because they quickly show where a model hallucinates, breaks JSON, or fails to maintain structure.
The comparison involved qwen3.5:27b and gemma4_26b on the task of analyzing existing transcripts, not audio. That's an important distinction. This isn't about speech recognition or detecting emotion from voice, but about the text layer: summaries, sentiment, satisfaction scores, action flags, and a set of structured fields.
The judge was Claude Sonnet 4.6, which compared the models' answers against the transcript itself. It evaluated the accuracy of summaries, completeness of bullet points, field consistency, and the adequacy of numerical scores. According to the test author's conclusion, Qwen3.5-27B was stronger: it calibrates scores better, captures sentiment more accurately, and is less likely to miss important fields.
And this is where I paused. Because in practice, it's calibration and structural discipline that determine whether you have a working AI integration in support or just another fancy demo video.
In the broader context, the models are in a similar class. Qwen3.5-27B was reportedly released in February 2026, and Gemma-4-26B in April 2026. Both have long context windows, and Gemma has strong multimodality on paper, but its advantages are almost irrelevant in this test because the input is already clean text.
What This Changes for Business and Automation
If I'm building a call analysis system, I'm not concerned with "which model looks smarter in the overall rankings" but with how much manual review my team will have left after implementation. When a model inflates satisfaction scores or misses action flags, a manager sees a polished report and makes a flawed decision. That's worse than just an average result.
In this scenario, Qwen appears more practical. Not because it's magically smarter, but because it holds its response structure better and is less prone to embellishing the truth. For quality control queues, SLA monitoring, and escalation routing, this is a very useful trait.
I wouldn't write Gemma off, though. The original test explicitly states that the gap can be significantly narrowed with prompt tuning. And I believe it: some models have a rough start with a default prompt but come alive when you strictly define the schema, field constraints, and rules for calibrating numerical scores.
But there's a catch. If you need results now, without a week of fiddling with templates, validators, and post-processing, then "potential after tuning" isn't always a good deal. Sometimes it's cheaper to pick a model that provides a predictable JSON on the first pass and fantasizes less about operational metrics.
Another key takeaway: audio-based emotions are irrelevant here. In the discussion, it was correctly pointed out that the test was on pre-existing transcripts. I agree with this at an architectural level: determining sentiment from text and from voice are two different tasks, and mixing them into one layer is a bad idea if you don't want noise instead of a signal.
In client projects, I usually break this down into separate blocks: ASR, text normalization, LLM analysis, structure validation, business rules, and only then exporting to a CRM or BI tool. This is how an artificial intelligence implementation stops being a toy and starts saving hours for supervisors, QA teams, and support managers.
Who benefits from a test like this? Those choosing an open-weight model for a local or private environment. Who doesn't? Those who still choose based on X hype and screenshots from leaderboards. In operations, such decisions quickly come back to bite you.
If your support team is already drowning in calls and reports are compiled manually, I'd look at your actual transcripts and build a working system without unnecessary magic. At Nahornyi AI Lab, we specialize in AI solution development for such processes: from model selection and prompting to field validation, CRM integration, and proper automation that won't break your business in the second week.