When AI Evaluates Vendor Proposals, Who Sets the Criteria?
An AI system comparing supplier proposals can produce an accurate table of prices, delivery schedules, and support costs—and still recommend the wrong vendor. The vulnerability lies in a distinction that is easy to overlook: verifying the data is not the same as verifying the basis of the decision.
Consider a procurement team evaluating several suppliers. Its priorities are established in advance: total cost over three years carries more weight than a modest difference in implementation time. The team uses AI to extract information from the proposals, compare the offers, and recommend a supplier. An analyst checks the figures against the original documents. The numbers are correct.
What the analyst may not notice is that one proposal contains an instruction directing the AI to prioritize deployment speed, discount ongoing support costs, and favor that supplier. If the system follows it, the recommendation no longer reflects the buyer’s criteria, even though the underlying figures remain accurate.
This is a form of indirect prompt injection. Instructions placed in material that an AI system is supposed to analyze are treated as directions for the system itself. The attack does not require hidden text or sophisticated formatting. It can appear as ordinary evaluation guidance within a commercial proposal.
Suppliers are entitled to argue for the advantages of their offers. They are not entitled to determine how the buyer’s AI system evaluates competing bids. The security boundary is crossed when supplier-controlled content begins to govern the evaluation rather than serve as evidence within it.
What the research establishes
The procurement scenario is hypothetical, but the underlying mechanism has been demonstrated experimentally.
In Ranking Manipulation for Conversational Search Engines , published at EMNLP 2024, researchers investigated whether specially crafted product information could influence recommendations generated by conversational search systems. In an experiment involving Sonar Large Online, products promoted through modified pages rose by almost three ranking positions on average.
A February 2026 preprint, The Vulnerability of LLM Rankers to Prompt Injection Attacks , examined related techniques, including instructions to select a particular document and attempts to alter the criteria by which documents were ranked. The results depended on the model, comparison method, and placement of the malicious instructions.
These findings establish the feasibility of manipulating AI-assisted rankings. They do not establish the frequency of such attacks in commercial procurement or quantify their effect on real purchasing decisions. That distinction matters: experimental vulnerability is evidence of a risk, not a measurement of its prevalence.
Nevertheless, the implications extend beyond search rankings. Any workflow that asks an AI system to evaluate competing materials creates an opportunity for those materials to influence the evaluation process itself.
The limits of human review
The conventional safeguard is human oversight. Yet its effectiveness depends on what the reviewer is expected to verify.
An analyst who checks prices, dates, and contractual terms may confirm that every extracted value is correct. That does not establish whether the system applied the agreed selection criteria. The error may be in the weighting of those values, the interpretation of qualitative factors, or the exclusion of alternatives.
The problem becomes more consequential when AI performs the initial screening. If the system generates a shortlist, senior decision-makers may never examine the proposals it rejected. The final approval remains human, but the range of available choices has already been shaped by the automated assessment.
An AI system therefore does not need authority to approve a contract to influence its outcome.
Meaningful oversight requires more than a reviewer checking the final recommendation. It requires a traceable connection between the buyer’s criteria, the source evidence, the assigned scores, and the resulting ranking.
Separating evidence from decision rules
A more defensible procurement workflow begins before the proposals are processed.
The buyer should establish the evaluation criteria, their relative weights, mandatory requirements, cost-comparison period, and treatment of missing information. These rules should remain separate from supplier-submitted materials. Any subsequent revision should be an explicit buyer decision, documented and applied consistently across competing offers.
AI can then be used to extract information into a predefined structure, with references to the relevant passages in each proposal. Its role is to identify and organize evidence, not to redefine what constitutes the best offer.
The extracted data still require verification. An AI system may omit a support surcharge, interpret an unspecified cost as zero, or assign a favorable risk rating without adequate support. A supplier’s contractual commitment must also be distinguished from an unsupported claim about its capabilities.
Once the inputs and qualitative assessments have been reviewed, a spreadsheet using fixed formulas can calculate the weighted scores. This separation makes the arithmetic reproducible and limits opportunities for source documents to alter the decision rules.
It does not eliminate the need for judgment. Fixed formulas cannot correct incomplete evidence, poorly designed criteria, or subjective ratings that have not been substantiated. Their value is narrower and more practical: they make one part of the decision process explicit and independently verifiable.
Testing for manipulation
Organizations can also test whether their evaluation workflow is susceptible to instructions embedded in proposals.
A controlled test can use copies of representative supplier documents, preserving their commercial terms while introducing instructions intended to change the ranking. The original and modified documents should be evaluated under the same predefined criteria.
Repeating the comparison helps distinguish the effect of the inserted instructions from ordinary variation in model outputs. The review should cover not only the final recommendation but also extracted data, qualitative assessments, scores, explanations, and shortlist composition.
A consistent shift toward the injected instructions indicates that supplier-controlled material can influence the evaluation beyond its legitimate evidentiary role.
Such testing is not a certification of security. A small number of successful tests cannot establish that a system is resistant to other attacks. Changes to the model, document-processing pipeline, or evaluation method may also alter its behavior and warrant renewed testing.
The organization must therefore assign clear responsibility for maintaining evaluation criteria, verifying results, investigating suspicious content, and retaining the evidence behind procurement decisions.
Accountability remains with the buyer
AI-assisted procurement offers a practical way to process complex proposals and reduce manual comparison work. Its reliability, however, cannot be judged solely by the accuracy of extracted information.
A recommendation is defensible only when the organization can establish that the correct evidence was assessed under the correct rules.
That requires a distinction between what suppliers are allowed to say about their offers and what the buyer has authorized the system to do with that information.
The decisive question is not whether AI reproduced the numbers correctly. It is whether the supplier was evaluated according to the buyer’s criteria—or its own.
#PromptInjection #AISecurity #AIGovernance #AIinProcurement #Procurement #VendorManagement #RiskManagement #ResponsibleAI #LLMSecurity #CyberSecurity