Understanding AI Prompts and Why Things Go Wrong

“Draw a cute character of a shark with pencil” (Generated by DALL-E)

Artificial intelligence (AI) has long transformed how we create art, enabling users to generate stunning visuals simply by describing their ideas, and I finally got around to try it, painfully. Apparently, this convenience comes with unexpected surprises. A straightforward request for a “cute cartoon shark” yield an image with pencils or even legs! Why does this happen? How can we fix it?

The Problem: When AI Misinterprets Prompts

According to ChatGPT, AI models rely on extensive training data, consisting of millions of images and descriptions, to generate art. This reliance often leads to unintended results because of:

1. Overgeneralized Associations.
   AI connects concepts it has frequently encountered together. For example, “pencil sketch” may bring to mind not only the sketching style but also the tools used to create it, like pencils or erasers.
2. Ambiguous Language.
   Vague prompts leave room for interpretation. A request for a “cute cartoon shark” doesn’t specify important details, such as whether additional elements or context should be excluded. Adding “with pencil” might add extra ambiguity to the prompt.
3. Scene Completion Tendencies.
   AI often tries to “complete” a scene. If it assumes something is missing, like a pencil to accompany a sketch, it might add that element to make the image feel whole. Aren’t they just wonderful?

Example: The Cute Shark Gone Wrong

It started with a simple prompt: “Draw a cute character of a shark with pencil.”

What went wrong:

• The shark had legs, as the anatomy wasn’t clearly defined in the prompt.
• The AI included pencils, and even erasers, seemingly associating “pencil sketch” with drawing tools.

This example highlights how ambiguous prompts can lead to unexpected results, making it clear that precision is crucial. I would argue about the legs, but I decided to move on.

“For crying out loud!”

How AI Interprets Prompts

According to ChatGPT, there are at least 3 things that AI considers when interpreting prompts:

1. Training Data Bias.
   If most “pencil sketches” in the training dataset include tools, the AI learns to associate them, even if they’re not explicitly requested.
2. Filling in the Gaps.
   Without clear instructions, the AI guesses what’s needed. This “scene completion” approach can lead to creative but unintended additions.
3. Misalignment of Context.
   Phrases like “cute cartoon shark” can lack critical details about anatomy, background, or additional elements, leaving the AI to interpret freely.

How to Avoid Bad Prompts

Based on my discussion with ChatGPT, here are common factors to avoid from crafting bad prompts:

• Ambiguity: Important details are missing, leading to misinterpretation.
• Overgeneralization: Includes broad terms like “sketch” without specifying the style or exclusions.
• Lack of Exclusions: Fails to rule out unwanted elements, like pencils or additional objects.

Takeaways

At this point, we simply need to remember 3 things:

1. Vague Prompts = Unintended Results.
   Ambiguous requests often lead to outputs that don’t align with our vision.
2. AI Relies on Patterns.
   The AI’s training shapes its output. If our request aligns too closely with common patterns in the dataset, we’re likely to get those patterns in return, whether we like it or not.
3. Precision is Key.
   Clear and detailed prompts help the AI focus on what we want and avoid unnecessary elements.

“At last, just how I want it!”

Conclusion

The journey of creating art with AI begins with understanding how it interprets prompts. By recognizing the pitfalls of ambiguous language, overgeneralized associations, and scene completion tendencies, we’re better equipped to craft requests that align with our vision.