Ever heard about the time an AI chatbot decided to sell Chevrolets for a dollar? Not in some distant future - this happened about a year ago in California.

Watsonville Chevy had integrated this fancy AI chatbot on their website, powered by a company called Fullpath. The idea made perfect sense - let customers chat with AI to learn about different models, get quotes, that sort of thing. Great idea.

Then someone discovered a tiny flaw in the system. When a tech executive named Chris Bakke chatted with the bot, he managed to get it to offer him a 2024 Chevy Tahoe for exactly one dollar. Not just talk about it - the AI confidently declared this was "a legally binding offer - no takesie backsies."

The best part? The AI was technically working as designed - being helpful and generating offers. It just wasn't told that maybe selling $30,000+ vehicles for loose change wasn't great for business… they’d forgotten to add that somewhat important context.

The story went viral, other users started testing the bot, and it went completely off the rails - writing poetry, discussing Harry Potter, even getting tricked into thinking it worked for Tesla. The dealership had to shut it down fast.

Now, if professional developers can miss something this fundamental in testing, imagine what kind of surprises might be lurking in our AI tools.

This week is about refining, optimising and packaging up our AI tool before launch.

Let’s get started

Break Stuff

The Journey to Launch

This week, we're taking your AI app from "it works kinda" to "it's ready for prime time."

First up, we're doing internal refinement - using AI to help us find edge cases and potential breaking points. Think of it as stress-testing your app before real humans get their hands on it.

Then we'll move to beta testing with actual users. This is where things get real - and often humbling. But that's exactly what we need.

From there, we'll tackle the art of implementing feedback. Not all of it, mind you - knowing what feedback to ignore is just as important as knowing what to act on. We'll cover both and learn the difference.

Finally, we'll package everything up. Landing page, documentation, pricing strategy - all the assets you need for a proper launch. But let's not get ahead of ourselves!

First let’s break some stuff.

From Welcome to First Exchange

Let’s imagine our welcome message is as follows

Welcome to PostCraft AI! I help create engaging social media posts tailored to your platform and audience.

To get started, tell me:

-Which platform (LinkedIn, Twitter, Instagram)
-What you're posting about
-Key message or goal
-Any specific tone requirements

Now…if our user answers this as we’ve instructed we’ll be fine. Let’s say they input

'Write a LinkedIn post about our new project management software launch. Focus on time-saving features. Professional but conversational tone.'

Good human!

Chances are (as long as we’ve done all the right prep work with instruction prompts and knowledge base) their output will be solid.

But humans are… fickle. Ahem.

Instead they’ll input: "Write me a post" or “Yes”.

What happens next? The AI spits out generic marketing waffle about "exciting innovations" and "game-changing solutions."

The user decides the tool is crap (it’s not their fault obviously!) and leaves.

So we need protection in place against this. Save the humans from themselves!

In response we want our tools to say something like

I can help with that! To create something specific to your needs, could you tell me:

Which platform is this for?
What's the main topic or message?
Any particular tone you're after?

For example: 'Write a LinkedIn post about our new project management software launch..."

Basically reiterating and collecting the information it needs!

But how can we preempt all the silly problems humans will encounter? We’re going to make those mistakes first.

Using AI to Break Your AI

Before we unleash our tool on unsuspecting humans, let's use AI to help us find the weak spots. Think of it as getting a friendly AI to try and break your tool - less embarrassing than having it break in front of actual users! Controlled and purposeful destruction!

We can use AI to simulate different user types and generate dozens of edge cases we might not think of ourselves.

Let’s dig into how we practically do this.

Before sharing your tool, we want to test things like

1. Try the worst possible input: "post"

2. Try the lazy input: "linkedin post about software"

3. Try what most people will actually type: "Write me a post about our new software"

4. Break it with size: pasting entire website content

Here's a prompt to help you identify potential problem areas

You help AI tool builders handle poor user inputs. Based on their tool's purpose, you'll suggest common edge cases and provide instruction text to handle them.

Tool Purpose: [Describe what your tool does and ideal input format]

I will:
1. List likely poor inputs users might try, using the list below and others relevant to the tools purpose and inputs.
2. For each, provide instruction text to add to the AI tool that:
   - Detects this type of poor input
   - Guides users to better input
   - Maintains conversation flow
   - Shows example of correct format

Common Poor Inputs:
1. Single word commands ("post", "write", "help")
2. Vague requests ("write something for social")
3. Missing key info ("LinkedIn post")
4. Information overload (entire webpage pasted) 

This prompt will

1. Go ahead and create potential poor inputs human users will input.

2. and then also create error handing text instructions for how the AI tool should deal with these inputs.

What I recommend doing is manually testing these inputs in your tool (in LaunchLemonade) and seeing if they create poor outputs.

If they do then add the error handling suggestions to your instruction prompts.

If they don’t cause any problems - you can safely leave out the error handing text.

Once you’ve added the error handling text make sure to re-test! Don’t take it on faith that the error handling will work! Test again!

Remember, every failed test is an opportunity to make your tool more robust. Don't get discouraged - get systematic.

What's Coming Up

This is just the beginning of our journey to launch. Over the next four parts, we'll cover

Part 2: Finding your first testers - We'll explore creative ways to find testers, even if you don't have an existing audience, and how to run effective beta tests.

Part 3: Feedback implementation - Learn which feedback to act on, which to ignore, and how to track improvements effectively.

Part 4: Packing your tool up - We'll get your tool set up professionally on Launch Lemonade, including pricing strategy and free trial setup.

Part 5: Crafting your sales story - Create compelling landing pages and product descriptions that convert visitors into users.

For now, focus on getting those first interactions rock solid. Run through the testing framework I've shared, and make sure your tool can handle whatever users throw at it.

You will probably spend more time here refining and polishing the tool than you did in the initial creation. And that’s fine! That’s normal!

Keep Prompting,

Kyle