Part 2: When Everyday Work Becomes AI Data

What Are We Teaching AI? — Part 2 of 2

A developer asks an AI assistant to help design a service.

The first proposal is reasonable. It is also wrong for the situation.

“We cannot introduce another database.”

The assistant revises the design.

“This needs to work with the existing deployment pipeline.”

Another revision.

“The support team must be able to investigate failures without accessing production data.”

By the time the conversation produces something useful, the developer has contributed much more than a prompt. They have described infrastructure constraints, operational practices, access boundaries, and the needs of another team.

Some of that knowledge might exist in documentation. Some might only exist in the experience of the people doing the work.

An ordinary conversation has made part of it explicit.

In Part 1, I looked at corrections and preferences: the small judgments we provide while asking AI to help us. This part looks at something broader—the context we supply so the help can become useful in the first place.

The answer depends on what we bring

A generic question usually produces a generic answer.

Ask how to build an authentication system, and an assistant can describe familiar patterns. Getting a useful answer for an actual product requires more detail.

What clients do we support? How do sessions expire? What happens when several requests try to refresh the same token? Which decisions are already fixed? What failures can the team tolerate?

The work of answering those questions often belongs to the user.

We explain the environment, reject assumptions, introduce exceptions, and describe what happened during earlier attempts. Each clarification narrows the space of acceptable solutions.

The resulting conversation can contain a detailed account of how a person or organization makes decisions.

That is why I think “user input” can be an incomplete description of what we contribute. Sometimes the input is a small piece of professional knowledge that took years to acquire.

Experience becomes easier to record

Consider the difference between these two instructions:

“Write a response to this customer.”

“Write a response that acknowledges the delay, avoids promising a delivery date we cannot verify, and offers a refund only if the order has not entered fulfillment.”

The second instruction contains a workflow and several decision rules.

The same pattern appears in other kinds of work:

• A designer explains why a visually attractive layout will confuse returning users.
• An editor identifies where technically accurate writing gives the wrong impression.
• An engineer describes a failure that only appears during deployment.
• A project manager explains which dependencies are negotiable and which are not.

These examples illustrate how practical knowledge can become part of an AI interaction. They do not establish what any particular provider does with that interaction afterward.

But they show why the material can be valuable. It connects a task to the conditions under which an answer would actually work.

A conversation is not a training pipeline

This is where the distinction from Part 1 matters again.

Sending information to an AI service allows that service to process it for the requested task. It does not, by itself, establish that the information will be used to train a future model.

Several separate questions need to be considered:

1. Processing: What information is needed to generate the response?
2. Retention: Is that information stored, and for how long?
3. Access: Who or what can access the stored information?
4. Evaluation: Can the interaction be reviewed to assess system quality?
5. Training: Is the interaction eligible for use in updating model parameters?

The answers depend on the product, account type, agreement, settings, and applicable policies.

A service can retain information without using it for model training. A tool can retrieve a document as context without incorporating it into model weights. An assistant can follow a rule within a conversation without permanently learning it.

Treating all of these as the same thing makes it harder to ask useful questions.

The fact that a system can use information now does not tell us everything about what happens to that information later.

Useful feedback still needs verification

Even when interactions are eligible for training, a conversation is not automatically a reliable lesson.

A user may describe an outdated process. A proposed solution may never have been tested. A confident correction may be incorrect. The final answer may have been accepted because a deadline was approaching.

Imagine a developer telling an assistant, “That fixed it.”

What does that establish?

Perhaps the code compiled. Perhaps one test passed. Perhaps the developer only checked the example that originally failed.

Without additional evidence, acceptance is a limited signal.

This is one reason the distance between collecting interactions and improving a model matters. Selection, interpretation, validation, and evaluation affect whether the material teaches useful behavior or reinforces mistakes.

Professional context can be valuable while still being incomplete.

The contribution is easy to overlook

From the user's perspective, the exchange is usually practical.

There is a task to finish. More context produces a better answer. Providing that context feels like the natural next step.

The interface rarely makes us stop and describe the contribution in another way:

“I am documenting how we handle this exception.”

“I am explaining the criteria we use to reject a solution.”

“I am turning an unwritten operating rule into text.”

Yet that may be exactly what is happening.

My concern is the gap between how small the interaction feels and how much knowledge it can contain.

That gap does not prove misuse. It does make transparency important. People should be able to understand whether their information is being processed for the current task, retained for later use, or made eligible for broader model improvement.

Those are meaningfully different choices.

Some knowledge is not ours to share alone

Workplace context also complicates the idea of individual consent.

A conversation may include a customer's message, a colleague's explanation, an internal incident, or a process developed by an entire team.

Having access to that information does not necessarily mean having permission to send it to an external service.

Before including it, I would ask whether the task can be explained with less detail.

Could an example replace a production record? Could names and identifiers be removed? Could the relevant constraint be described without uploading the entire document?

Removing names is not always enough. A distinctive incident or a combination of details may still identify the people or organization involved.

The goal is to preserve the context needed for useful assistance while making a deliberate decision about what leaves the original environment.

Make the exchange visible

I do not think the answer is to stop giving AI tools context.

Context is often what makes them useful. Explaining a problem clearly can also improve our own understanding and produce documentation we should have written earlier.

What I want is a more conscious exchange.

For users, that means understanding the rules of the service they are using and recognizing when a prompt contains someone else's information or a valuable internal process.

For teams, it means agreeing on which tools and data are appropriate, instead of leaving every employee to interpret the boundary alone.

For providers, it means explaining data use in language that distinguishes immediate assistance from retention, evaluation, and training.

There is also a practical habit worth keeping: when an AI conversation helps uncover an important rule or decision, bring that knowledge back into the team's own documentation. Review it, test it where possible, and give it an accountable owner.

A useful explanation should not disappear into a chat history.

We bring our standards, constraints, and experience to these tools because we want better results. That contribution deserves to be visible.

Before asking only what the AI produced, we can also ask:

What did we have to teach it about our work to make that result possible?

---

Previous: You Think You’re Asking. You’re Also Teaching.