How do you balance friendly AI support responses with actually closing tickets quickly?

We replaced our standard canned responses with AI-written drafts to speed up support. It definitely cut down our initial response time, but it brought an unexpected problem: users are opening way more follow-up tickets.

Because the AI sounds so natural, customers treat it like a friendly conversation rather than a quick sign-off, which keeps our queue packed. Now we're re-tuning our prompts to focus on closing tickets instead of just being overly conversational.

Have you had to tweak your support AI to make sure it actually finishes conversations instead of extending them?