A Pragmatic Guide for Customers and Suppliers
As artificial intelligence becomes embedded in ordinary software products and services — from CRM tools and marketing automation suites to cybersecurity platforms — many companies are discovering that traditional software contract provisions no longer fit the underlying technology. Some still read as though drafted for either (i) a science fiction dystopia, or (ii) a 1998 on-prem software license.
In-house legal teams representing buyers of AI-enabled services — or the suppliers building them — are expected to strike a balance between innovation, governance, and practical risk allocation. Starting with a three-page “AI Prohibited Activities” exhibit often makes that harder, particularly when the provisions bear little resemblance to what the underlying technology actually does.
This two-part series lays out a practical framework for negotiating AI-related terms. Part 1 covers how to identify what kind of AI is actually in scope and the issues that matter most for customers. Part 2 covers the supplier’s side and a short set of drafting principles for both.
Different AI Technologies Present Different Considerations
Before negotiating, it helps to know what kind of AI functionality is actually in scope — because the risk profile, and the contract language it calls for, changes considerably depending on the answer.
Traditional machine learning — rule-based systems, predictive analytics, classification models trained on structured data — typically does not generate new content. It is trained on curated datasets and improved through usage data over time. The primary risk is accuracy, secondary risks include bias and data protection.
Generative AI (including agentic AI) — systems built on large language models, diffusion models, or other foundation models that generate new text, images, code, or summaries — generally carries a different set of risks entirely: hallucinations, IP infringement claims, confidentiality leakage, attorney-client privilege waivers, training data provenance, and ownership and usage of outputs.
Customer-facing vs. Embedded. A chatbot or drafting assistant the customer’s users interact with generally requires clear terms on accuracy, output ownership, usage guidelines, user-facing disclosures, and data ownership and usage of outputs. An LLM used internally — to classify tickets, generate summaries, or assist support teams, for example, with no customer-facing exposure — can behave more like traditional SaaS data processing models, with a greater focus on data protection and security considerations rather than bespoke output warranties.
Public vs. Private is a distinction most standard contract language still doesn’t acknowledge. A public instance such as an API-based multi-tenant model may process prompts and metadata in shared infrastructure; whereas a private or dedicated instance, sometimes fine-tuned for the customer, is often contractually restricted from training on that customer’s inputs. The difference changes what protections are realistic to negotiate.
In light of these differences, the contractual considerations associated with each model also tend to differ in meaningful ways.
Top Issues for Customers
-
Use of Customer Data for Training
Pre-AI software agreements rarely addressed this question at all — there was no “training” to speak of, only processing. Today, it is often the most heavily negotiated point in an AI contract.
Customers want to protect their confidential and personal data from being used to train broader models. That raises a few key questions:
- What customer data, if any, can be used for generalized model training or improvement?
- Does the answer change for confidential information, personal data, anonymized data, or aggregated usage data?
- Should telemetry, diagnostics, security monitoring, and performance optimization be treated differently from foundation model training?
Market practice is converging on three distinctions. Identifiable, or even anonymized, customer data cannot be used to train models accessible to the public or to third parties. Customer data may be used only for models or customizations dedicated to that customer alone. Aggregated and de-identified data is generally available for service improvement, though not for foundation model training.
Better contracts draw a clear line between “permissible use of data to operate and improve this service” and “use of data to train a generalized model.” Broad, undifferentiated restrictions on all forms of learning can end up limiting a supplier’s ability to improve accuracy, performance, or security — which is not what most customers actually want.
-
Output Ownership and IP Risk
Traditional software licenses assumed a direct connection between input and output; the software did what it was told, and ownership questions were comparatively simple. Generative AI breaks that assumption: outputs are probabilistic, shaped by training data the customer never sees, and the line between “the customer’s work product” and “the supplier’s underlying model” is genuinely blurry.
Customers generally want to obtain sufficient rights to use AI-generated outputs while minimizing IP infringement risk. That raises a few key questions:
- What rights does the customer need to use, commercialize, or rely upon outputs generated from its inputs?
- How do those rights interact with the supplier’s underlying IP in the platform, reporting templates, and pre-existing training materials?
- How do customer inputs, supplier technology, and training datasets affect ownership of, or rights to use, outputs?
- Which risks sit within the supplier’s control (e.g., those arising from the underlying platform or model architecture) and which arise from customer behavior (e.g., prompts or modifications) or downstream uses outside the intended scope of the service?
Market practice is converging on three positions. Customers generally receive ownership of, or broad rights to use, outputs generated from their own inputs, subject to the supplier’s IP in the platform, models, and standard templates. Suppliers retain pre-existing IP and any improvements that do not incorporate customer-specific information. Supplier indemnities cover claims arising from the supplier’s core technology, with carve-outs for claims traceable to the customer’s own prompts, modifications, or out-of-scope use.
-
Accuracy and “Hallucination” Risk
This is perhaps the clearest example of an old contract concept — the error-free or “conforms to specifications” warranty — meeting a technology it was never built for. An absolute accuracy warranty for a generative AI system is not just hard to negotiate; it’s often not a meaningful description of how the technology works.
Because customers want to reduce the risk of business decisions being made on inaccurate, incomplete, or misleading output, a few questions help frame the issues:
- What level of performance is reasonably achievable given the underlying technology?
- What disclosures about known model limitations, confidence levels, or intended use cases are appropriate?
- Which use cases warrant human review or approval workflows, and which party is best positioned to validate outputs before they are used in business-critical decisions?
The emerging approach has three components. Performance is tied to documented specifications rather than absolute accuracy. Suppliers commit to transparency about known limitations and material changes to model architecture. Human review or approval workflows are built in for high-risk or business-critical use cases.
The shift is from “guarantee the outcome” to “be transparent about the limitations and put the right oversight in place.”
In the next installment (Part 2), we will review the supplier’s side of these same issues, as well as a short set of drafting principles that apply regardless of which side of the table you’re on.
Jason Karp has more than 30 years of experience in the telecom, technology, media, XaaS, and public safety industries. Jason works with businesses of all sizes, handling a wide range of complex commercial and corporate transactions, as well as business operations and market strategy, corporate governance, compliance program development and implementation, and regulatory and policy advocacy and strategy.