The data
Seven tiny pictures, each handed over as a string of text
- Four shapes a model could recognise: a 1, a 7, a cross and a circle.
- Three blocks of random noise. Nobody can guess random pixels, so an exact answer proves the file was truly decoded.
- Kept tiny on purpose. At 8×8 the text is 20–130 characters, so decoding by hand is tedious and not hopeless. Caveats ↓
What the model actually received. One picture of a 7, stored five ways, easiest to hardest. Each string is exactly what was pasted in.
The picture: a 7, 8×8 pixels →
Results
One model decoded everything. Another came close.
Each tile is the number of images a model rebuilt with every pixel right. Click any tile to see every image behind it.
Reasoning is the whole difference.The same model on the same files goes from 0 decoded with reasoning off to exact answers with it on. The switch ↓
The weak answers were not cautious ones.Gemini 3.8 Flash missed 16 images and gave 10 of those with high confidence. Examples ↓
Cost
What being right costs
Each point is one model at one reasoning setting. Up is more accurate, left is cheaper. Every gridline is ten times the cost of the one before it.
Seven images per format per point, except GPT-6 Luna with reasoning off and at low effort, which ran on four. With all formats selected, each point averages its five formats equally. The dashed line joins GPT-6 Luna’s three settings, from reasoning off to high effort.
Not obvious from the numbers
- GPT-6 Sol is the best buy. The only perfect score, at about 60% of Sonnet’s cost.
- GPT-6 Luna costs a quarter of what Sol does and decodes two thirds of the images. It is solid on the simple formats and fades on BMP and PNG.
- The cheap model fails by running out of room, not by being wrong. When Luna finished a PNG it was usually right. More ↓
- Gemini 3.8 Flash at low effort costs as much as Sol and decodes about half. Ten of its sixteen misses came with high confidence.
- PNG is the expensive format. About 45 seconds and 4–7 cents for Sol and Sonnet, against seconds and under a cent for the same picture as PBM.
- Decoding is not recognising. GPT-6 Luna rebuilt the cross exactly, then called it “a white diamond”. More ↓
Bottom line. A strong model with reasoning on can rebuild a small image from its raw bytes. That is a real capability, and it is not a better way to send images: attaching one normally is far cheaper and far more reliable. Read with care ↓
That is the result. Everything below is the how.Optional reading · the full story, the method, the caveats, the bill
The full story
Ten steps from a hunch to a result
We turned tiny images into strings of letters and digits, pasted them into four language models as plain text, and asked what the picture was. No tools, no code, no vision. This is the story of what happened, including the parts where nothing did.
1 The hunch
A picture is a file, and a file can be written as text
When you attach an image to a chat, the model looks at it with a separate vision system. But an image file is only a sequence of bytes, and any bytes can be spelled out in ordinary characters. That spelling is called base64, and it looks like gibberish.
Models read base64 text surprisingly well. So the hunch was: could a model read an image that way too, working backwards from the gibberish to the picture by reasoning alone? It is like being handed a page of Morse code and asked what the drawing is.
See one picture stored five ways, exactly as the models received it ↑
2 The first try, and the objection
It seemed to work. That was the problem.
The first round was done by hand, pasting the PNG of the 7 into chat apps. Two of five answers were pixel-perfect.
Then my manager pointed out the hole in it:
A chatbot is an agent. It has access to code. It can save that string, convert it to an image, use its vision, and read it. You do not want that.
He was right. A correct answer from a chat app proves nothing, because the app may have quietly run a decoder. Everything had to be redone where cheating is impossible.
| Chat app, by hand | Answer |
|---|---|
| Claude Opus 5.5 43 s | The digit 7. Every pixel right. |
| Gemini 3.8 Flash, extended thinking 5 min 50 s | The digit 7. Every pixel right. |
| ChatGPT about 50 s | Right pixels, colours swapped. Could not name it. |
| Gemini 3.8 Flash, instant | “A hollow circle.” High confidence. |
| Gemini 3.6 Flash, instant | “The letter C.” High confidence. |
Suggestive, and worthless as evidence. Nobody could say whether a tool had been used.
3 Take the tools away
Text in, text out, nothing else
Every run from here on goes through the API with no tools attached. The model receives characters and returns characters. Whatever it gets right, it worked out in its own reasoning.
STEP 1
Draw a tiny image
Black and white, 8 pixels by 8. Small enough that decoding by hand is tedious and not hopeless.
STEP 2
Store it five ways
From a plain grid of 0s and 1s up to a compressed PNG. Same picture, very different amounts of work.
STEP 3
Demand every pixel
The model must write out the whole grid before naming the picture, so a lucky guess scores nothing.
The test images
Four shapes and three blocks of random noise. The noise is the real test: nobody can guess random pixels, so a model that reproduces them exactly has truly decoded the file. The seven images are at the top ↑
4 The pilot
One image, ten answers
The first API calls sent every model the PNG of the 7. Here is what came back, drawn exactly as each model wrote it. A red dot marks a wrong pixel.
3 modelsdecoded a compressed PNG with nothing but reasoning. The objection was answered.
“A smiley face”Gemini’s answer to a picture of a 7, with high confidence and zero reasoning.
0 tokensof reasoning from Claude Sonnet 5.5 at low and medium effort. It only thought when set to high.
5 The switch
Reasoning is the whole difference
GPT-6 Luna is the cheapest model here, so it was the one to sweep across reasoning settings: four images, five formats, reasoning off, low and high.
With reasoning off it scored 0 of 16 on the encoded images, with pixel accuracy of 38–67%. That is what flipping a coin gives. Turn reasoning up and the same model, on the same files, starts producing exact decodes.
Each cell is exact decodes out of four images. The text grid needs no decoding, so it is right every time.
| GPT-6 Luna | Off | Low | High |
|---|---|---|---|
| Text grid | 4 | 4 | 4 |
| PBM | 0 | 2 | 4 |
| PGM | 0 | 0 | 4 |
| BMP | 0 | 4 | 2 |
| PNG | 0 | 0 | 5 of 12 |
Some models bluff when they don’t think
A classic pixel-art smiley face. Confidence: high.Gemini 3.8 Flash, no reasoning. The image was a 7.
One of them refused to
A made-up grid would be fabricated, so I’m not giving one.Claude Sonnet 5.5, no reasoning
6 The wall
Then the cheap model ran out of room
On PNG, GPT-6 Luna did something new. It did not answer wrongly. It did not answer at all. It reasoned until it hit its 12,000-token limit and returned an empty reply, 15 times out of 16.
So the limit was doubled to 24,000 and every failed call was sent again. That rescued some of them: 6 of the 15 came back pixel-perfect. Eight ran out of room a second time.
When Luna did finish a PNG, it was usually right: 8 of the 10 grids it produced were pixel-perfect. Its main failure was running out of room, not getting it wrong.
23calls that hit a limit and returned nothing
$0.19paid for those empty replies, and nothing to show for it
An empty reply is billed like any other. This was the most expensive lesson of the project.
Why some PNGs were easy
A PNG can pack its pixels in different ways. Luna breezed through both PNGs that store their pixels uncompressed, even at four times the size, and struggled with the ones that use real compression.
| PNG | Compression inside the file | GPT-6 Luna, high effort |
|---|---|---|
| X, 16×16 | None | Exact, 256 of 256 |
| Noise A, 16×16 | None | Exact, 256 of 256 |
| 7, 8×8 | Fixed Huffman | 2 of 3 runs exact |
| X, 8×8 | Fixed Huffman | 2 of 3 runs exact |
| Noise A, 8×8 | Fixed Huffman | 1 of 3 runs exact |
| Noise B, 8×8 | Fixed Huffman | 0 of 3, out of tokens |
| Noise B, 16×16 | Fixed Huffman | Out of tokens |
| 7, 16×16 | Dynamic Huffman | Gave up after 22,790 tokens |
The compression type was read from the files. That it explains the pattern is an inference from these eight.
7 The full test
One model decoded everything. Another came close.
The final round put all four models through all five formats on seven images: four shapes and three blocks of noise. A cell counts only when all 64 pixels are right.
35 of 35GPT-6 Sol. Every image, every format, including three compressed PNGs of random noise.
33 of 35Claude Sonnet 5.5. It got all the noise PNGs and missed two shape PNGs by two and three pixels.
0 of 7PNGs for Gemini 3.8 Flash at low effort, five of them answered with high confidence.
The full table, with every image behind every cell, is at the top ↑
8 The price of being right
Accuracy against cost
Each point is one model at one reasoning setting. Up is better, left is cheaper. The cost axis is logarithmic: every gridline is ten times the one before it.
PNG is where the money goes: about 45 seconds and 4–7 cents a call for Sol and Sonnet, against seconds and under a cent for the same picture as PBM. The chart and what it shows are at the top ↑
9 One more twist
Decoding is not recognising
Getting every pixel right and knowing what the pixels show turned out to be two different skills.
- GPT-6 Luna decoded the X exactly, then called it “a white diamond on a black background”.
- It decoded the 7 exactly and described “a short horizontal bar with a zigzagging stem”.
- GPT-6 Sol decoded all seven PNGs, but four of them with black and white swapped. PNG stores black as 0 and our prompt asked for black as 1, so that trap was partly ours.
10 What it means, and what it doesn’t
Yes, they can. No, you shouldn’t.
A strong model with reasoning switched on can take the raw bytes of a small image, decompress them in its reasoning, and reproduce every pixel. That is a real capability, and it is not vision.
This is not a better way to send images. Attaching an image normally is far cheaper and far more reliable. What this measures is how far reasoning alone can go on raw data.
Read with care
- Small counts. Seven images, one run per cell. Enough to see the pattern, not enough for fine comparisons.
- Tiny images. These are 20–130 characters of base64. A real photo is hundreds of thousands. Nothing here says the result carries over.
- Limits cut both ways. A model that hit its token limit did not finish; that is not the same as being unable.
- Unequal settings. Each model has a different range of reasoning effort, so each is shown where it reasons.
The bill
Run it yourself
Every image, prompt, raw reply and score is in the repository. Replies are cached, so re-running costs nothing unless you add something new.
python make_images.py # test images python make_prompts.py # encodings and prompts python run_openrouter.py --phase 1 --dry-run python score.py # replies to results.csv python make_site.py # this page
Repository · The full story, with the technical detail · Results report · Every raw reply