One of our key takeaways from the report was that “vendors who build and typically host GenAI models do not provide a complete set of controls that mitigate these risks. Instead, users need to acquire solutions that do so to augment the vendors’ limited controls.”
We’re proud that Harmonic Security is listed as a Representative Vendor in this guide.
What is GenAI TRiSM?
According to Gartner, the market forGenerative AI in Trust, Risk and Security Management (GenAI TriSM) “is a subset of a larger AITRiSM market, which includes multiple software segments that can only be implemented by the builders or owners of AI models, applications or agents. These builders or owners create, maintain and govern these entities.Builders or owners of AI models are sometimes the same entities that use or interact with those same AI models. In the case of large foundation models, the users of the models are typically different entities than the ones who build or own the models.”
Again, according to the report GenAI TRiSM specifically “comprises multiple software and services segments that support security, data protection and risk mitigation for adopters of GenAI applications and model interactions.”
Content Anomaly Detection
The guide outlines three areas of GenAI TRiSM:
Content anomaly detection. Includes
- "Unacceptable or malicious use"
- “Enterprise content transmitted through prompts or other methods that results in compromise of confidential data inputs
- "Hallucinations or inaccurate, illegal, copyright-infringing and otherwise unwanted outputs that compromise enterprise decision making”
Data protection. Includes
- “Data leakage and compromised confidentiality in hosted vendor environment or internal, self-managed environments”.
- "Inability to govern privacy and data protection policies in externally hosted environments
- "Difficulty conducting privacy impact assessments and complying with various regional regulations due to the“black box” nature of the third-party models
AI application security. Includes
- "Adversarial prompting attacks”
- "Vector database attacks"
- "Hacker access to model states and parameters."
Specifically around content anomaly detection, Gartner recommends the organizations “use content anomaly detection products that mitigate input and output risks to enforce acceptable use policy and prevent unwanted or otherwise illegitimate model completions and LLM applications responses from compromising your organization’s decision making, safety and security.”
Harmonic Security Named Representative Vendor for Content Anomaly Detection
GenAI TRiSM tools include solutions for:
- Content anomaly detection
- Data protection
- AI application security
These tools complement associated TRiSM measures implemented by builders or owners of GenAI models, applications and agents.
Gartner recommends “Use content anomaly detection products that mitigate input and output risks to enforce acceptable use policy and prevent unwanted or otherwise illegitimate model completions and LLM applications responses from compromising your organization’s decision making, safety and security.”
Learn More about Harmonic Security
If you’re curious about GenAI TRiSM and want to better protect your sensitive information from leaking into LLMs, we’d love to hear from you!
Harmonic Security has been selected as a ‘finalist’ for the RSA Conference 2024 Innovation Sandbox Contest, which means you can come along on May 6th and see what we’ve been up to!
To learn more about Harmonic, or to arrange to meet us at RSA, contact us here: https://www.harmonic.security/book-a-meeting.
Gartner, Innovation Guide for GenerativeAI in Trust, Risk and Security Management, By Avivah Litan, Jeremy D'Hoinne,Gabriele Rigon, 12 April 2024
GARTNER is a registered trademark and service mark of Gartner, Inc. and/or its affiliates in the U.S. and internationally and is used herein with permission.All rights reserved.
Gartner does not endorse any vendor, product or service depicted in its research publications, and does not advise technology users to select only those vendors with the highest ratings or other designation. Gartner research publications consist of the opinions of Gartner’s research organization and should not be construed as statements of fact. Gartner disclaims all warranties, expressed or implied, with respect to this research, including any warranties of merchantability or fitness for a particular purpose.
.png)