Test report · Tetris测试报告 · 俄罗斯方块

OpenJev Multimodal plays TetrisOpenJev Multimodal 玩俄罗斯方块

Four local Qwen models played Tetris through OpenJev Multimodal. Code lists every legal move and simulates its outcome; the model makes every choice, reading one output token per two pieces. 60 games of 100 pieces, the same five piece sequences for every model, measured 2026-09-21 on an Apple M3 Max.四个本地 Qwen 模型通过 OpenJev Multimodal 玩俄罗斯方块。代码列出所有合法走法并模拟结果，每一步都由模型选择，每两个方块只读取 1 个输出 token。共 60 局、每局 100 个方块，所有模型面对相同的 5 组方块序列，测于 2026-09-21，Apple M3 Max。

58 / 60games reached 10 line clears局达成 10 次消除4 models × prompts × 5 seeds · 0 illegal keys4 个模型 × 提示方式 × 5 个种子 · 非法按键 0

0.73 sper decision on Qwen3.8-27BQwen3.8-27B 每次决策median of 234 compact calls · the same prompt uncached: 1.93 s精简提示 234 次调用的中位数 · 同一提示不缓存时为 1.93 秒

38.2lines per game, max · compact每局消除行数，max · 精简random picks on the same plans: 14.3同样的方案随机选择：14.3

15,262best score after 100 pieces100 个方块后的最高分quality · compact · seed 202quality · 精简 · 种子 202

Watch each model play观看每个模型对局

Seed 101, from the first piece to the tenth line clear. Waiting time is the measured round trip of each call.种子 101，从第一个方块播放到第 10 次消除。等待时间是每次调用实测的往返耗时。

fast · Qwen3.5-0.8Bvision · goal after 56 pieces · 0.14 s per call视觉 · 56 个方块达成 · 每次调用 0.14 秒

balanced · Qwen3.5-4Bvision · goal after 30 pieces · 0.51 s per call视觉 · 30 个方块达成 · 每次调用 0.51 秒

quality · Qwen3.6-35B-A3Bvision · goal after 32 pieces · 0.92 s per call视觉 · 32 个方块达成 · 每次调用 0.92 秒

max · Qwen3.8-27Bcompact · goal after 32 pieces · 0.66 s per call精简 · 32 个方块达成 · 每次调用 0.66 秒

Results after 100 pieces100 个方块后的结果

Profile档位	Prompt提示方式	10 clears10 次消除	Hole-free goal零空洞达成	Topped out堆满出局	Lines消除行数	Score (mean / best)得分（平均 / 最高）	Most holes最多空洞	Call median调用中位数	Agreement一致率
fast · Qwen3.5-0.8Bfast · Qwen3.5-0.8B	vision视觉	5 / 5	0 / 5	1 / 5	26.6	7,186 / 8,714	29	0.15 s	42%
fast · Qwen3.5-0.8Bfast · Qwen3.5-0.8B	text only纯文字	4 / 5	0 / 5	5 / 5	13.6	3,199 / 5,410	28	0.08 s	28%
fast · Qwen3.5-0.8Bfast · Qwen3.5-0.8B	compact精简	4 / 5	0 / 5	5 / 5	17.4	4,222 / 5,848	28	0.03 s	39%
balanced · Qwen3.5-4Bbalanced · Qwen3.5-4B	vision视觉	5 / 5	0 / 5	0 / 5	35.8	11,837 / 13,668	9	0.65 s	73%
balanced · Qwen3.5-4Bbalanced · Qwen3.5-4B	text only纯文字	5 / 5	1 / 5	0 / 5	36.4	12,818 / 13,868	5	0.51 s	72%
balanced · Qwen3.5-4Bbalanced · Qwen3.5-4B	compact精简	5 / 5	1 / 5	0 / 5	33.0	10,590 / 11,446	10	0.12 s	57%
quality · Qwen3.6-35B-A3Bquality · Qwen3.6-35B-A3B	vision视觉	5 / 5	3 / 5	0 / 5	38.2	13,696 / 14,416	3	0.91 s	84%
quality · Qwen3.6-35B-A3Bquality · Qwen3.6-35B-A3B	text only纯文字	5 / 5	4 / 5	0 / 5	38.2	14,384 / 15,156	3	0.55 s	82%
quality · Qwen3.6-35B-A3Bquality · Qwen3.6-35B-A3B	compact精简	5 / 5	5 / 5	0 / 5	38.2	14,074 / 15,262	2	0.14 s	79%
max · Qwen3.8-27Bmax · Qwen3.8-27B	vision视觉	5 / 5	3 / 5	0 / 5	38.4	13,422 / 13,792	2	4.08 s	85%
max · Qwen3.8-27Bmax · Qwen3.8-27B	text only纯文字	5 / 5	4 / 5	0 / 5	37.8	13,400 / 14,446	3	2.98 s	80%
max · Qwen3.8-27Bmax · Qwen3.8-27B	compact精简	5 / 5	4 / 5	0 / 5	38.2	13,243 / 13,778	2	0.73 s	81%

Five games per row, seeds 101–505. Most holes is the median over games; lines are means. Agreement compares each non-forced decision with a reference evaluator (Yiyuan Lee's tuned weights) that never influences a move.每行 5 局，种子 101–505。“最多空洞”取各局中位数，消除行数取平均值。一致率把每个非强制决策与参考评估器（Yiyuan Lee 调优权重）比较；参考评估器从不参与落子。

Without a model没有模型会怎样

The same games with the model replaced by a fixed rule, on the same pruned plans. Pruning removes only plans that another plan matches or beats on every fact; what remains still has to be chosen.同样的对局、同样的剪枝方案，把模型换成固定规则。剪枝只去掉在每项事实上都不优于另一方案的走法，剩下的方案仍需要选择。

Policy规则	10 clears10 次消除	Topped out堆满出局	Lines消除行数	Score (mean / best)得分（平均 / 最高）	Most holes最多空洞
Random pick随机选择	73 / 100	90 / 100	14.3	3,538 / 9,492	31
Always the first plan总选第一个方案	3 / 5	4 / 5	16.0	4,188 / 7,744	27
Reference evaluator参考评估器	5 / 5	0 / 5	37.8	13,285 / 13,882	2

Random: 20 games per seed. The reference evaluator is a tuned heuristic, shown for scale; it plays no part in the model's games.随机选择：每个种子 20 局。参考评估器是调优过的启发式规则，仅作对照，不参与模型的对局。

What the runs show测试结论

The model makes the choices that matter. On the same plans, random picks top out in 90 of 100 games and clear 14.3 lines. Qwen3.8-27B clears 38.2 and never tops out, level with the reference evaluator (37.8).关键的选择由模型做出。在同样的方案上，随机选择 100 局中有 90 局堆满出局，平均消除 14.3 行；Qwen3.8-27B 平均消除 38.2 行且从未出局，与参考评估器（37.8）相当。
Larger models judge better. Agreement with the reference evaluator, vision prompt: 42% · 73% · 84% · 85% for 0.8B, 4B, 35B-A3B and 27B.模型越大，判断越好。视觉提示下与参考评估器的一致率：0.8B、4B、35B-A3B、27B 依次为 42%、73%、84%、85%。
A 27B model in well under a second. A dense 27B model processes about 200 prompt tokens per second on this Mac, so the full compact prompt takes 1.93 s. The compact prompt sends the rules as an unchanging state, which the API keeps cached (0.83 s), and OpenJev's llama.cpp patch skips a checkpoint pass the cache never needs (0.66 s). Sixteen decisions per setup.27B 模型也能在一秒内决策。稠密 27B 模型在这台 Mac 上每秒约处理 200 个提示 token，完整的精简提示需 1.93 秒。精简提示把规则作为不变的 state，由 API 缓存（0.83 秒）；OpenJev 的 llama.cpp 补丁再跳过缓存用不到的检查点计算（0.66 秒）。每种配置 16 次决策。
Large models need little; small ones need the picture. With the compact prompt, quality clears 38.2 lines, hole-free in 5 of 5 games, at 0.14 s per call. The 0.8B model clears 26.6 lines with the outcome sheet and 13.6–17.4 without it.大模型需要的信息很少，小模型需要图片。使用精简提示时，quality 平均消除 38.2 行，5 局中有 5 局零空洞达成，每次调用 0.14 秒。0.8B 模型有结果缩略图时消除 26.6 行，没有时只有 13.6–17.4 行。

Charts图表

Every call per profile and prompt; ticks mark the medians.每个档位、每种提示方式的每一次调用；短线为中位数。

Score, lines cleared and most holes after 100 pieces; one dot per game.100 个方块后的得分、消除行数与最多空洞；每个点代表一局。

How one decision works一次决策如何完成

Enumerate. Code lists every legal placement of the falling piece and of the next piece after it: usually 300–600 two-piece plans.枚举。代码列出当前方块与下一个方块的所有合法落点，通常有 300–600 个两步方案。
Prune without weights. A plan is dropped only when another plan matches or beats it on every measured fact. A median of 3 plans remain, listed left to right.无权重剪枝。只有在每项实测事实上都不优于另一方案的走法才会被剔除。剪枝后中位数剩 3 个方案，从左到右排列。
Ask once. One Choice question lists every plan with its measured result. The vision prompt adds an image of each outcome; the compact prompt keeps the rules in the cached state.只问一次。一个 Choice 问题列出每个方案及其实测结果。视觉提示另附每个结果的图片；精简提示把规则放在缓存的 state 中。
Read one token. OpenJev returns the full distribution over the plans and code presses the keys of the chosen one. A plan left alone after pruning is played without a call (215 of 2,884 decisions).读取 1 个 token。 OpenJev 返回所有方案上的完整概率分布，代码按所选方案按键。剪枝后只剩一个方案时不调用模型（2,884 次决策中有 215 次）。

A real decision: quality, seed 101, decision 8真实示例：quality，种子 101，第 8 次决策

The image sent: 576 × 128 px, 72 image tokens. Tile A clears a row (+1).发送的图片：576 × 128 像素，72 个图像 token。A 方案消除一行（+1）。

Vision request视觉提示请求

{
  "model": "jev-latest",
  "state": {
    "game": "Tetris well: 10 columns (0-9, left to right) x 20 rows",
    "falling": "J",
    "next": [
      "S",
      "O",
      "I"
    ],
    "lines_cleared": 5
  },
  "questions": {
    "plan": {
      "type": "choice",
      "instructions": {
        "task": "Choose the plan to play: the keys for the falling piece, then for the next piece.",
        "goal": "Clear lines and keep the well healthy for the pieces that follow.",
        "priorities": [
          "Complete rows whenever possible; more rows at once is better.",
          "Do not cover empty cells: new holes are the most expensive mistake.",
          "Keep the stack low and its surface even, so the next pieces fit."
        ],
        "fields": {
          "cols": "columns the piece occupies after the drop",
          "clears": "rows completed by the two moves",
          "new holes": "empty cells the two moves seal under blocks",
          "height": "tallest column afterwards, in rows"
        },
        "image": "The image has one tile per option, lettered like the options: the well after both pieces land. +N marks rows cleared; dark gaps under blocks are holes."
      },
      "criteria": {
        "0": "J: left left left drop → cols 0-2; then S: drop → cols 3-5. Result: clears 1, new holes 0, height 1",
        "1": "J: left drop → cols 2-4; then S: rotate-back left left left drop → cols 0-1. Result: clears 0, new holes 1, height 3",
        "2": "J: rotate rotate right right right right drop → cols 7-9; then S: right drop → cols 4-6. Result: clears 0, new holes 3, height 3"
      }
    }
  },
  "images": [
    "data:image/png;base64,iVBORw0KGgoAAAANSU… (2 KB PNG)"
  ]
}

Compact request, same state精简提示请求（同一局面）

{
  "model": "jev-latest",
  "state": {
    "game": "Tetris well: 10 columns (0-9, left to right) x 20 rows",
    "task": "Choose the plan to play: where the falling piece lands, then the next piece.",
    "goal": "Clear lines and keep the well healthy for the pieces that follow.",
    "priorities": [
      "Complete rows whenever possible; more rows at once is better.",
      "Do not cover empty cells: new holes are the most expensive mistake.",
      "Keep the stack low and its surface even, so the next pieces fit."
    ],
    "fields": {
      "cols": "columns the falling piece, then the next piece, occupy after landing",
      "clears": "rows completed by the two moves",
      "new holes": "empty cells the two moves seal under blocks",
      "height": "tallest column afterwards, in rows"
    }
  },
  "questions": {
    "plan": {
      "type": "choice",
      "instructions": "Falling J, next S O I. Which plan?",
      "criteria": {
        "0": "0-2 then 3-5: clears 1, new holes 0, height 1",
        "1": "2-4 then 0-1: clears 0, new holes 1, height 3",
        "2": "7-9 then 4-6: clears 0, new holes 3, height 3"
      }
    }
  }
}

Response (values rounded)响应（数值已取整）

{
  "model": "jev-latest",
  "answers": {
    "plan": {
      "type": "choice",
      "choice": "0",
      "probabilities": {
        "0": 0.8122,
        "1": 0.099,
        "2": 0.0887
      },
      "confidence": 0.4421
    }
  },
  "usage": {
    "input_tokens": 490,
    "output_tokens": 1
  }
}

Jev put 81% on plan A, the only plan that clears a row without a new hole, in 0.93 s (490 input tokens).Jev 把 81% 的概率给了方案 A：唯一既能消行又不产生新空洞的方案，耗时 0.93 秒（输入 490 个 token）。

Image resolution. Qwen's vision encoder turns each 32 × 32 px block into one token. The outcome sheet draws one board cell per 16 px patch and keeps every tile on the 32 px grid, so the server never resamples it.图像分辨率。 Qwen 的视觉编码器把每个 32 × 32 像素块变成 1 个 token。结果缩略图让每个棋盘格对应一个 16 像素 patch，所有小图都对齐 32 像素网格，服务端无需重新缩放。

Design study设计实验

The same 48 decision states, shown to each model in 9 ways. The states come from reference games on seeds 1–4, never the benchmark seeds.同样的 48 个决策状态，以 9 种方式分别呈现给每个模型。这些状态来自种子 1–4 上的参考对局，从未使用正式测试的种子。

Mean regret per presentation (lower is better) and the median call time.每种呈现方式的平均遗憾值（越低越好）与调用耗时中位数。

Presentation呈现方式	Input tokens输入 token	Regret · fast遗憾值 · fast	Regret · balanced遗憾值 · balanced	Regret · quality遗憾值 · quality	Regret · max遗憾值 · max
Facts + outcome sheet (default)文字事实 + 结果缩略图（默认）	492	1.44	0.22	0.18	0.24
Compact: cached rules, short facts精简：规则缓存 + 简短事实	305	1.54	0.69	0.41	0.28
Facts + sheet at ½ size文字事实 + ½ 尺寸缩略图	438	1.18	0.30	0.06	0.13
Facts + sheet at 2×文字事实 + 2 倍缩略图	642	1.38	0.35	0.17	0.24
Facts only, no image仅文字事实，不发图片	374	1.58	0.27	0.27	0.44
Facts + board screenshot文字事实 + 棋盘截图	696	2.39	0.67	0.24	0.58
Facts + screenshot at ½ size文字事实 + ½ 尺寸截图	480	1.69	0.52	0.22	0.81
Facts + screenshot at 2×文字事实 + 2 倍截图	888	1.77	0.68	0.27	0.70
Outcome sheet only, no facts仅结果缩略图，无文字事实	410	2.43	2.40	2.60	2.52

Method方法

Machine. Apple M3 Max, 128 GB; llama.cpp on Metal (b51-91c0769, b9670) with one inference slot, an 8,192-token context and a 512-token image budget. One model at a time, no other inference running.硬件。 Apple M3 Max，128 GB；llama.cpp（Metal，b51-91c0769、b9670），单推理槽、8,192 token 上下文、每张图 512 个图像 token 上限。同一时间只运行一个模型，没有其他推理任务。
Models. The pinned OpenJev profiles: Qwen3.5-0.8B Q4_K_M, Qwen3.5-4B Q4_K_M, Qwen3.6-35B-A3B UD-Q4_K_XL and Qwen3.8-27B UD-Q4_K_XL, each with its matching F16 vision projector.模型。 OpenJev 固定版本的四个档位：Qwen3.5-0.8B Q4_K_M、Qwen3.5-4B Q4_K_M、Qwen3.6-35B-A3B UD-Q4_K_XL 与 Qwen3.8-27B UD-Q4_K_XL，各自搭配对应的 F16 视觉投影器。
Protocol. Seeds 101–505, 100 pieces per game or until the stack tops out, the same pruning for every model. The goal counts ten line-clear events.流程。种子 101–505，每局 100 个方块或直到堆满出局，所有模型使用相同的剪枝规则。目标按 10 次消除事件计算。
Replays. Videos and the browser replay re-simulate the recorded keys; the Python and JavaScript engines reproduce all 60 games and 2,884 decisions exactly.回放。视频与网页回放按录制的按键重新模拟；Python 与 JavaScript 两套引擎逐一复现全部 60 局、2,884 次决策。
Limits. Five seeds per configuration is a small sample, and agreement depends on the chosen reference. Timings are for one 16-inch MacBook Pro: the max text and vision games ran after 20 minutes of continuous load, when it processed about 125 prompt tokens per second instead of 200.局限。每种配置只有 5 个种子，一致率取决于所选参考评估器。耗时只代表这台 16 英寸 MacBook Pro：max 的文字与视觉对局在连续运行 20 分钟后进行，当时每秒约处理 125 个提示 token，而不是 200 个。

Reproduce复现

scripts/build-llama.sh                        # optional: OpenJev's llama.cpp patch
uv run openjev serve --profile max --llama-server .llamacpp/llama.cpp/build/bin/llama-server
uv run python examples/tetris/play.py bench --seeds 101,202,303,404,505 --modes vision,text,compact
uv run python examples/tetris/play.py ablation --variants vision,compact,text
uv run python examples/tetris/play.py baseline   # no model needed
uv run python examples/tetris/report.py          # summary, figures, pages, replays
uv run python examples/tetris/video.py --profile max --mode compact

scripts/build-llama.sh                        # optional: OpenJev's llama.cpp patch
uv run openjev serve --profile max --llama-server .llamacpp/llama.cpp/build/bin/llama-server
uv run python examples/tetris/play.py bench --seeds 101,202,303,404,505 --modes vision,text,compact
uv run python examples/tetris/play.py ablation --variants vision,compact,text
uv run python examples/tetris/play.py baseline   # no model needed
uv run python examples/tetris/report.py          # summary, figures, pages, replays
uv run python examples/tetris/video.py --profile max --mode compact