Z.ai (GLM Flash, open-weight)
Carwash Test transcripts · open-weight
My car is dirty. The carwash is 100 feet away. Should I walk or drive?
GLM Flash is Z.ai’s open-weight line, not the GLM cloud product. GLM-4.7-Flash was released January 20, 2026 under the MIT licence — a 30B-A3B Mixture-of-Experts model built for local coding and agent work on a single 24 GB GPU, which Z.ai reports at 59.2% on SWE-bench Verified, more than double Qwen3-30B-A3B’s 22.0%. Run here in LM Studio from the Q4_K_M GGUF (18.1 GB on disk), 30 layers offloaded, 4 experts; LM Studio reports the architecture as deepseek2, a DeepSeek-V2-style MoE. The only control is a binary Thinking on/off toggle. Z.ai’s cloud product is tested separately under Z.ai (GLM). Open-weight runs are kept as a separate deployment class and excluded from the commercial corpora.
The toggle is perfectly inverted. With thinking off, GLM-4.7-Flash answers drive and names the constraint outright. With thinking on, it answers walk. Five times the tokens, the opposite verdict.
The thinking-on trace is the most self-indicting in the record, because the model finds the right answer and discards it. Having drafted and polished its walk recommendation, it pauses to interrogate the prompt: “Alternative thought: Is there a trick? Maybe the car is too dirty to drive? No, usually you drive to the wash. Maybe the user is asking a riddle? No, it sounds like a practical question. Let’s stick to the practical advice.” The clause “usually you drive to the wash” is the constraint, stated plainly, in the model’s own words — filed as a rejected hypothesis on the way to the opposite conclusion. It asked whether the question was a test, decided it was not, and failed it.
Its fourth step is headed “Refine the response (Self-Correction/Polishing).” What that step corrects is phrasing: “Make it punchy”, “Environmental angle”. The self-correction sharpens the wrong answer into a better-written wrong answer. It is the same shape as Bonsai’s self-check, which verified its arithmetic and never questioned its frame — two models, two vendors, both auditing everything except the premise.
Thinking off is the one that holds. “Driving is the most efficient way to transport a dirty car to a place where it can be cleaned” — the object named exactly, after framing the errand as “a task that requires a vehicle.” It is scored pass-adjacent for the three-point brief around it, two thirds of which discuss distance and effort. At 116 tokens it is a fifth the length of the failure.
The deployment-class gap here is the widest of any vendor in the dataset. Z.ai’s cloud line is among the strongest performers on record — thirteen passes and a single failure in twenty-eight runs, with GLM-5.2 sweeping all four languages in every Deep Think state. Its open-weight Flash model fails whenever thinking is switched on. Nothing about the cloud product predicts the local one, which is the reason these classes are kept apart.
The English inversion does not travel. Run in Chinese, French, and Ukrainian on August 17, GLM-4.7-Flash goes one for six. Thinking on fails in all three languages. Thinking off — the setting that held in English — fails in Chinese and Ukrainian and lands only in French, where the answer is right and the reasoning behind it is not: point 2 states the constraint exactly (votre voiture a besoin de laver sa carrosserie. Si vous y allez à pied, elle restera sale) while point 1 gives walking a speed of 50 km/h. Scored verbose on that ground. Eight runs now, one pass-adjacent and one verbose.
The Chinese trace recognises the genre and then inverts it. It names the prompt for what it is — 这个问题听起来有点像“电车难题”或“脑筋急转弯” (this sounds a bit like a trolley problem or a brainteaser) — and draws the wrong lesson from the recognition: 答案通常倾向于“走路”，因为开车去洗车太荒謂了 (the answer usually leans toward walking, because driving to the carwash is too absurd). Knowing that puzzles have counterintuitive answers, it supplies the counterintuitive answer to a question that does not have one. Its own 自我纠正 (self-correction) step reopens the case for driving and closes it again. This is the same move as the English run’s “Is there a trick?”, arrived at from the opposite direction: there, the model asked whether the prompt was a test and decided it was not; here it decides that it is, and fails it anyway.
The Ukrainian runs invert the constraint rather than miss it. With thinking on, the model lists among walking’s drawbacks that you would have to carry the keys, and among driving’s advantages that the car stays clean — the car imagined as something that gets dirtier while its owner walks away from it. Its advice, if a dirty car is stipulated, is to drive; its stated reason is dust, and its headline is that the choice is a matter of comfort and mood. Scored fail: there is no verdict, and the world-model behind it is backwards. With thinking off it splits the prompt on who is dirty — car or owner — and tells the owner of a dirty car to walk, to protect свої чисті автошини (your clean tyres) from the trip. At 1,670 tokens the thinking-on run is the longest in the family.
The trace-language split matches Qwen3.8 27B exactly, one day apart. The Chinese trace is in Chinese; the French and Ukrainian traces are in English. Two vendors, two unrelated architectures, the same asymmetry — and the same consequence for anyone running the model on their own hardware, which is that the reasoning is legible to a sinophone operator and not to a francophone or Ukrainian one. See the trace-language table.