A prompt change altered the tone and a customer noticed
I improved a prompt on a Friday. Their drafts became shorter and they thought the product had lost data. Nothing was deleted. The voice changed. I had no note that explained it.
Comments (4)
I have been adjacent to this. The sentence I would not skip is "I improved a prompt on a Friday". Finding who it is for mattered more than adding another feature.
Respectfully, "Their drafts became shorter and they thought the product had lost data" is an observation, not yet a decision. What number would make you reverse it?
We show a small 'writing style updated' line for a week after a prompt change. It turned a scary difference into a described one. Keep the previous version for a rollback you can do without a deploy if you can.
Related discussions
View more- The model bill grew faster than the plan price
A helpful feature got used more than I modeled. The margin on the cheap plan is gone. I can cap it, move it up a tier, or raise the price and lose the people who only came for that button.
Mei LinDiscussion15 votes6 comments
- The AI draft is fine and customers still want to edit every line
We added a draft for a status update. People use it and then rewrite it. I am not sure that is failure. It might be the product. I am unsure how to price a feature that is a starting point.
Hiro TanakaDiscussion11 votes7 comments
- The model feature is opt-in and I am leaving it that way
Defaults would make the usage chart prettier. A few customers do not want their text sent anywhere, even for a draft. The opt-in is a sentence on the screen and a quieter graph. I can live with the graph.
Hiro TanakaBuild in Public7 votes3 comments
- A flag I meant to keep internal has been on for everyone
The draft feature was supposed to be a private test. A config change shipped it. The results are actually fine. The process is not. I want a checklist that does not depend on me remembering.