You spend hours hammering away, screwing stuff in, swearing and sweating.

And can’t get the damn thing up.

Would you throw it away? Just because you can’t get it done immediately? Smash the bugger up and throw the pieces out?

Nah. Well, I hope not!

Same goes for your tone of voice tool. It might not be perfect immediately. It probably won’t be! Do you drop the project?

Think about it: If you hired a new copywriter who got your voice 80% right on their first assignment, would you immediately fire them?

Of course not. You'd give them feedback. You'd help them understand what wasn't quite landing. You'd train them.

The same principle applies to your AI tone of voice system. That first output isn't the final product—it's the starting point of an iterative training process. Just like a new team member, your AI needs guidance, feedback, and time to really nail your voice.

Let's get started

Setting Realistic Expectations

First up: your initial outputs won't be perfect. This isn't a failure of the process—it's a natural part of it.

Even the most sophisticated AI systems need training and refinement. Hell, that’s what makes them so good! What we're building is a feedback loop, not a one-time setup. The magic happens through iteration.

It’ll probably look a bit like this

• First round: 60-80% accuracy - a bit meh

• After targeted feedback: 80-90% accuracy - ok workable.

• After multiple iterations: 90-95% accuracy - wow this is solid!

• Ongoing refinement: 95%+ accuracy - BINGO!

The best news? This improvement happens quickly. Quicker than with humans!

The Two-Phase Training Approach

We'll tackle training in two distinct phases

1. Offline Testing: Controlled environment, simulated tasks

2. Online Implementation: Real-world application with monitoring

This approach lets you refine in a safe space before deploying your voice system in actual production scenarios.

When to make a switch? If you are building this for yourself then you’ll likely go live a lot earlier. If building for a client then you’ll do more testing and refining up front. It depends!

Phase 1: Offline Testing

Offline testing means giving your system tasks that mimic real-world needs but aren't actually used in production. It's like a dress rehearsal or a dry run. So many good metaphors here! Or similes? You know what I mean!

To properly test your voice system, you need a diverse set of challenges. Here's a prompt to help you generate appropriate some test tasks to start with:

You are an AI testing specialist. I need to create a comprehensive set of test scenarios for my brand voice AI system. The system will be used for [describe your intended use cases].

Please create 10 diverse test tasks that will thoroughly evaluate the AI's ability to replicate my brand voice across different contexts, tones, and content types.

For each test task:
1. Provide a clear prompt I should give to my AI
2. Explain what specific aspects of voice this test evaluates
3. Include a baseline for success (how to know if it passed)

Make sure the test set includes:
- Different content lengths (short, medium, long)
- Varying emotional tones (positive, neutral, challenging)
- Different content types (based on my intended use cases)
- Edge cases that might be particularly challenging

The goal is to identify where the AI excels and where it needs improvement in capturing my brand voice.

Use this prompt to generate a comprehensive set of test tasks tailored to your specific needs.

Now - what to do with these tasks?

Here's the exact process for systematically improving your custom instructions

1. Provide your current custom instructions and one of the test tasks to the AI

2. The AI completes the task based on those instructions

3. You give specific feedback on what wasn't right about the voice

4. The AI updates your custom instructions based on your feedback

5. You implement the updated instructions and repeat

We are basically stress testing our tone of voice AI with tasks. And each time telling it what it did well and what it sucked at. From that we get new custom instructions to work with.

This creates a tight feedback loop where each iteration improves your voice system.

Here's a prompt you can copy and paste to start this refinement process

You'll help me refine my input custom instructions for better voice accuracy. Follow these steps:

1. First, I'll provide my current custom instructions and a test task as inputs.
2. You'll complete the test task using those instructions. Complete the task only and provide no additional information. 
3. After seeing your response, I'll provide feedback
4. You'll then update my input custom instructions based on that feedback

When I provide feedback, analyse it carefully to identify patterns and issues with the voice. Then create updated custom instructions that address these issues.

Present the updated instructions in a clear, formatted way that I can easily copy and paste into my system. Use the same basic  formatting and structure as the input - just with amendments based on my feedback. Do NOT explain the changes - provide only the reworked custom instructions with no additional information

INPUTS begin:
Current Custom Instructions:
[PASTE YOUR CURRENT CUSTOM INSTRUCTIONS HERE]

Test Task:
[ENTER A TASK YOU WANT THE AI TO COMPLETE]

This prompt is a little complex. All you need to know is you copy and paste in your current custom instructions which we created in Part 2 and one of the tasks from above.

Run all of this in your Project, CustomGPT, API model or otherwise deployed tool so it has full access to your knowledge base.

This will run a process and spit out a reworked set of custom instructions.

Take these new custom instructions and add them into your deployed model.

Then run another task. And another. And another.

Each refinement cycle typically takes just a few minutes but dramatically improves your results. Most voice systems require 3-5 cycles to reach excellent accuracy. It’s well worth the investment of time.

When you're satisfied with the refinement, simply copy the final updated instructions into your voice system configuration and you as done for now!

Systematic Feedback Approach

Now comes the crucial part: providing effective feedback. Here's what makes feedback effective:

Poor feedback: "This doesn't sound right. Try again."

Effective feedback: "This is too formal for my brand voice. My writing typically uses contractions (I'm vs. I am), shorter sentences, and more conversational transitions. The paragraph structure is good, but the word choice feels stiff. Reference my blog posts from March for better examples of how I discuss technical topics."

The key elements of effective feedback

1. Be specific about what's off

2. Provide examples of what works better

3. Reference specific documents in your knowledge base

4. Focus on patterns, not just individual words

If in doubt imagine you are a teacher providing writing feedback to a student. Keep it simple and focused.

Phase 2: Going Live

Once your offline tests are consistently producing good results, it's time to implement your voice system in real-world scenarios.

Start small

1. Begin with lower-stakes content (internal documents, draft emails - depending on your use case obviously)

2. Have a human review outputs before they go public(!!!!)

3. Gradually expand to more visible channels

4. (Optional) And remove human checking as you progress and feel comfortable.

Even in production, the training doesn't stop. Here are some pointers though, especially if deploying for a client.

1. Regular Reviews: Schedule weekly or monthly reviews of AI outputs. These reviews can become more infrequent over time until you can more or less forget about them.

2. User Feedback: If multiple people use the system, collect their impressions

3. Content Audit: Periodically compare AI content against human-created content

4. Knowledge Base Updates: Add new examples as your brand evolves. Ideally automatically - ie. scrapes of the website on a scheduled cron job.

5. Model updates: when a new version of the model drops make sure that the tone of voice tool still works. And (if you want) update to the newer model. Rerun the refinement process each time as different models will react differently.

Wrapping up

We've covered a lot of ground in this 5-part series. Let's recap the entire journey

Part 1: We explored why brand voice matters in the AI age, examined the limitations of traditional approaches, and identified the massive opportunity for AI-powered voice systems.

Part 2: We learned how to gather, organise, and prepare voice examples across different content types, creating the raw materials for our voice system.

Part 3: We extracted the “essence” of your voice by creating powerful prompts that analyse your content and create custom instructions that capture your unique patterns.

Part 4: We looked at different ways to deploy your voice system, from simple personal setups to team-wide deployments, helping you choose the right approach for your needs or those of your clients.

Part 5: We looked at how to finalise our model and get it ready for launch, establishing a systematic training approach to refine your voice system through iterative feedback.

This project, whilst not a trivial amount of work (!), will leave you with a super powerful daily use tool. Whether you're using it to scale your own content creation or offering it as a service to clients you’re in a great place.

Keep Prompting,

Kyle