the piinc blog

There is no such thing as “the best” generative model for editing. Image editing covers a broad spectrum of tasks with entirely different use cases, and correspondingly, different user expectations. An image editing task can entail removing a small defect from an image or could involve completely re-imagining a scene. What users expect of a generative image editing model for these tasks are entirely different outcomes, yet the common tendency in model evaluation and benchmarking is to collapse all cases into a single ranked set of models or objective functions.

What’s in a generative model? A lot of judgement calls. These come through the training data that gets curated, labeled, and ingested; the reinforcement learning techniques and reward functions used to tune it; the mitigations that keep models in safe territories; and the prompt rewriting, reasoning loops, and inference-level parameter adjustments that are sometimes surfaced in the product and other times hidden from view. All these product and engineering decisions come together to mold the personality of the model that ends up in front of users. And these personalities start to show their faces once models are probed repeatedly across a broad set of subjects.

For the past few months, Piinc has been pushing new generative media models to their limits with image and video requests drawn from real creative workflows. Our goal is to evaluate whether models produce truly useful outputs: production-ready assets that need minimal to no further editing. Can these models understand user intentions out of the box, or do they need continuous prodding to reach the intended outcome? We’re not measuring whether outputs are generally likable – we’re assessing whether a model can be a good creative partner. And a good creative partner understands the user’s intentions and acts on them.