If you had to make a text-only LLM reason about images, but you weren't allowed to use a vision encoder, where would you look?

If you had to make a text-only LLM reason about images, but you weren't allowed to use a vision encoder, where would you look?