How to Train an AI Agent to Catch Filler Words

I talk a lot. Calls. Recordings. Thinking out loud with agents. Somewhere along the way I got tired of hearing myself stall.

You cannot improve what you do not notice. So I keep a short catalog of speech patterns and ask my AI setup to flag them when they show up. Not to be precious about language. To tighten the signal.

This page is the catalog and the how-to. Use it if you coach yourself, or if you help someone else sound clearer.

The patterns that count

These are the ones I treat as official. Count them. Do not shame them.

• Um / Uh — pure pause fillers. Harmless in small doses. Loud when they stack.
• "You know" — only when it is a stall tag ("so, you know, I just think…"). Not when it means something ("you know me").
• Sentence-initial "So" — the transition crutch. Mid-sentence "that is so cool" is fine. Starting every thought with "So…" is a tell.
• Mid-thought stutters and long trailing pauses — restarting, repeating ("the the way"), or drifting off mid-sentence and climbing back on. That is the brain buffering out loud.
• Trailing "Right?" — the rhetorical check-in glued to the end of a statement. Once you hear it, you cannot un-hear it.

Watch list

Not official yet. They keep showing up.

• Trailing "etcetera" as a list-exit hatch instead of a real "and more like this"
• Paragraph-initial "Okay" — same energy as sentence-initial "So"
• Mid-sentence "right," — the same word as a check-in in the middle of a sentence, not at the end

Promote a watch-list item only after you have seen it more than once. One flag is a coincidence. A pattern is a habit.

The "So" punctuation trap

Dictation software often inserts a period in the middle of a clause. Then it capitalizes the next word. You get:

…for this user. So I do not want to begin with inbox triage.

Read it without the fake period:

…for this user, so I do not want to begin with inbox triage.

That second "so" is an ordinary connective. It is not a discourse restart. Test before you count: delete the period, lowercase the "s". If the result is one grammatical sentence, do not count it.

This rule saves a lot of false positives if you work from transcripts.

How to train an agent to catch them

1. Give the agent the catalog. Official list first. Watch list second.
2. Tell it to flag, not to rewrite your personality.
3. Require a count and one short example per pattern. No essay.
4. Exclude meta talk. If you are discussing the catalog itself, do not count the examples.
5. Apply the "So" punctuation test on transcripts.

The point is not perfection. It is catching the autopilot. When the agent pings a pattern, notice, reset, and keep talking. Over enough reps, the cheap habits get quieter.

Clearer speech is better UX for whoever is listening. Including future-you listening back to the recording.

FAQ

What speech patterns should an AI agent flag?

Start with um, uh, stall-tag "you know", sentence-initial "So", mid-thought restarts, and trailing "Right?". Add a short watch list. Promote a watch-list item only after you see it more than once.

How do I reduce filler words without sounding stiff?

Flag them. Do not perform shame. Reset and keep talking. The goal is less autopilot, not a speech that sounds like a manual.

Should "you know" always count as a filler?

No. Count it when it is a stall tag. Do not count it when it carries meaning, as in "you know me".

Why does my transcript show so many sentence-initial "So"s?

Dictation often breaks one clause into two sentences. Run the punctuation test before you count. If removing the period yields one grammatical sentence, skip it.

Can I use this catalog with clients?

Yes. These patterns show up across speakers. That is why a small shared checklist beats starting from zero. Extend it with whatever is idiosyncratic to that person.

Sources

• This catalog is from live coaching work, not from a single public paper. I built it from observed instances, then reused it as a starting checklist.
• Related reading: public filler-word research in presentation coaching. I do not copy those lists. The official/watch-list split and the "So" punctuation test are the parts I added from transcript work.