The workflow I've used in the past for this is using the fantastic StableDiffusion 1.5 ControlNet lineart model. See https://x.com/P_Galbraith/status/1716299002969469054 for example.
Probably the ideal workflow would use a reference image via something like IP-Adapter since a simple colour palette wouldn't really give enough control (see https://x.com/P_Galbraith/status/1716405163420963196 for example). Typically you have the character design done in a flat perspective. So it would be nice to have something like an IP-Adapter input as well as a detailed line drawing and a rough paintover.
You would also need to provide a way to give lighting control (i.e. control over light sources and direction) as well as multiple characters etc... for it to be useful in complex compositions.
Would take a look at that for some inspiration -- The UI is Apache 2.0 and used by professional artists. I'd be curious how you think it performs relative to the workflow you've developed.
You're spot on that researchers don't always build the UI that end-users want to use. Always love to see people thinking about the creatives. Good work!
Regarding the interface Petalica gets it right, although mask + palette makes sense.
A fully local model with support with different styles and a configurable respect for the line art (with default 100% respect) would definitely be a game changer. At their current level, the AI models are slightly lacking, so it feels like too much time investment compared to the quality of the results.
[0] https://petalica.com/index_en.html [1] https://github.com/lllyasviel/style2paints/issues/235