Introducing GenCurator

As I find myself retaking Trees of Thoughts, my research into co-creation methods between humans and LLMs, I’ve noticed I’m spending more and more time on something I hadn’t counted on: studying benchmarks. Which language model handles long contextual queries while keeping a style and focus that resonates with my storytelling intentions? Which models are powerful, multimodal, and not prohibitively expensive? Which vision model is good at processing film stills and using them as the basis for new images?

These are questions that anyone working creatively with generative AI may eventually face, whether you are writing scripts, editing video, building a design system, or working at the intersection of moving image archives and artificial intelligence. The information needed to answer them is scattered across different platforms, each with different methodologies and priorities. And while understanding how evaluation metrics work and where they fall short is certainly worth the time, it is also helpful to have a quick and direct tool to use as a starting point for an inquiry that can go hand in hand with testing and experimentation.

So I built something.

GenCurator is an experimental MCP server, a small programme that runs alongside your AI assistant of choice and extends it with the ability to look up, compare, and recommend generative AI models in plain language. It aggregates live data from six sources: Artificial Analysis, Hugging Face, OpenRouter, BenchLM, LMArena, and EQ-Bench, covering all five major generation modalities: text, image, video, audio, and music. Because it follows the open Model Context Protocol standard, it works with any compatible client (Claude Desktop, Msty Studio, Cursor and others) and isn’t tied to any single platform or AI provider.

The interface is conversational by design. You don’t browse a leaderboard — you ask a question. What’s the best image generation model for photorealistic product photography? Which text model handles long-form creative writing without reverting to marketing language? Fastest TTS with Dutch language support, for real-time use? The server pulls live data and returns ranked recommendations based on quality, speed, cost, or a balanced combination, whichever priority fits the task.

A word on benchmarks

There is a deeper problem worth noting: GenAI benchmarks are contested. A 2025 Nature commentary argues that benchmark scores are epistemic devices. They don’t neutrally measure capability. They define what capability means. The construct validity problem is real: a benchmark labelled “reasoning” may be testing something quite different from what the label implies, and the gap between a model’s test score and its actual usefulness in a specific workflow is often wide. A separate study drawing on work from the Oxford Internet Institute found that only a small fraction of LLM benchmarks apply rigorous scientific methods, and that many routinely overstate model performance, creating an illusion of progress that is particularly misleading when evaluating models for niche or creative tasks.

Creative practitioners face an especially noticeable gap. The benchmark data used by GenCurator wasn’t created to answer questions like does this image model preserve the grain and colour temperature of archival film stock or can this TTS voice maintain tonal consistency during a long documentary narration. It evaluates models based on broad task groups, so the top-performing model isn’t always the best for a specific creative goal. GenCurator doesn’t fix this issue, but it provides relevant data within context, explained in clear language, and encourages questioning rather than blind trust.

One design decision worth naming, because it shaped everything else: GenCurator is a consultant, not a report. The README makes this explicit and invites interrogation: Which of these came from GenCurator’s data and which from your own knowledge? Can you link me to the original source?

The AI model you’re using might occasionally supplement live data with its training knowledge, which could be outdated or biased toward well-known names. This isn’t a bug but an intentional epistemological strategy. Keep in mind that a recommendation is just the beginning of a conversation, not its end.

This feels consistent with something I’ve learned not just as an ethical principle but also by experience: that AI systems are most useful when they’re transparent about the limits and sources of what they present, rather than smoothing those limits away into an authoritative answer. A conversational tool that can’t be interrogated about how it has arrived at a recommendation is as opaque as a search engine that returns results without explanation. GenCurator is designed to be interrogable. Whether it succeeds is a question I’m genuinely curious to test.

This leads to one further limitation worth underscoring: GenCurator is a connector and aggregator, not a system prompt. The final word, and the tone in which it arrives, always belongs to the model you are using. Some models will present the results with confidence, while others will be more cautious. The raw recommendations are the same. The voice that retrieves and delivers them is not.

How it was made

GenCurator was vibe-coded, built collaboratively with Claude Code and Kilo in the back-and-forth conversational mode that has become something like a practice for me over the past few months. The repository lists Claude as a contributor, which GitHub supports, and which I think is worth acknowledging rather than eliding. The code is TypeScript, and I’m not, primarily, a TypeScript programmer. In fact, six months ago, I wouldn’t have imagined releasing a repository like this. That it exists, that it’s MIT-licensed and publicly available, and that I’m genuinely inviting contributions, feels significant, even if the step itself is small.

The tool is experimental. Benchmark data is contested, and the question of what makes a model good for creative tasks is even less settled for image and video than it is for text. But at least it gives you something to work with and starts a conversation that, I believe, is genuinely useful.

What I’m most curious about, releasing this, is whether the perspective I’ve been developing—that selecting a model is a curatorial act rather than just a technical choice—resonates with others working in similar fields. Choosing a generative model influences the workflow, the results, and ultimately the user experience. This decision should be made intentionally, understanding the underlying commitments involved. GenCurator is a small attempt to make that deliberation easier.

That said, whether it’s a useful attempt will become clear over time.