Aspen Digital

Understanding generative AI’s risks and limitations

Before beginning to use GenAI, it is important to understand its limitations and associated risks.  Our ability to control and monitor the following aspects of model implementation is key to our ability to exercise responsible AI practices.

Foundational GenAI models are built on a very large pool of training data, but these sources are still finite.  As a result, that data, its uses, and outputs may:

• Not be accurate.  Outputs may be inaccurate, unverifiable, and possibly out of date.  This type of erroneous output includes overly confident responses not merited by training data, sometimes referred to as AI “hallucinations.”

• Infringe third party intellectual property rights.  Due to the way GenAI creates outputs based on the data available to it, results could lead to intellectual property rights infringements.

• Breach privacy rights.  Outputs may violate privacy and data protection laws.
• Include data which is biased or discriminatory.  This may lead to outputs which in turn amplify that bias.
• Disclose confidential information.  Data provided in prompts is used to train the model and can be incorporated into the model itself.  This could result in the disclosure of confidential information.
  • As a result, it is important to understand the full scope of our data classification policies, and what distinguishes “confidential” from “public” information.

The data issues that can occur with openly available models generally do not exist with our internal GenAI models, which are typically built for a specific purpose and which we can manage.  Therefore, while this technology brings great potential, when using openly available models you must consider the risks to our business and reputation in every engagement. 

Guiding principles

As you begin to use GenAI you will need to:

• Identify existing policies, processes, tools and frameworks that apply to the use of GenAI, including in the area of corporate security, technology, privacy, data handling, copyright and third-party risk management.
• Develop a standardized risk assessment framework that incorporates risks due to limitations of the technology as well as the risks of applying the technology in your specific business process, as you would with any third-party software or tool.[2]
• Upskill teams so they are able to detect issues unique to GenAI, such as concept drift and hallucinations.   
• Implement governance measures that address key areas of risk such as cybersecurity, privacy, legal, compliance, and regulatory risks; and consider establishing a Governance Council to facilitate multi-stakeholder engagement and provide senior oversight over the use of GenAI.
• Protect proprietary data and intellectual property via technical and contracting mechanisms.
• Monitor the use of the technology for compliance with evolving laws as well as policies and procedures we have implemented.   
• Learn how GenAI works.  Several publicly available sources explain the concepts behind GenAI and how it operates, and having a basic understanding of these principles will help you ask the right questions when working with these technologies.[3]
• Realize that GenAI may not necessarily be the right tool for every task or problem.  It’s okay to decide that GenAI is not appropriate in certain situations. 
• Ask questions of your peers, supervisors, and leaders and share your knowledge with them.  Many companies, including ours, are creating new policies, procedures, and infrastructure to guide their use of GenAI in a manner that preserves data privacy, security, confidentiality, and intellectual property—and we all benefit from ongoing dialogue and questions.

[2] You can consider models such as the National Institute of Standards and Technology’s AI Risk Management Framework, https://www.nist.gov/itl/ai-risk-management-framework, or the International Organization for Standardization’s guidance on managing AI risks, ISO23894, https://www.iso.org/standard/77304.html. 

[3] Aspen Digital’s emerging technologies team has developed a new primer on this topic, Intro to Generative AI, https://www.aspendigital.org/report/intro-to-generative-ai/.

Dos

1. Conduct queries and searches multiple times, and in multiple ways, to understand variance in outputs.
2. Create training, tutorials, and images for internal purposes, based on textual instructions (e.g., for inspiration in brainstorming, team meetings, internal presentations).
3. Debug code using test samples not sourced or derived from company code.
4. Experiment with code creation capabilities to learn how GenAI can create code and to evaluate the potential utility of such code.
5. Expand perspectives and new ideas for inspiration and brainstorming.
6. Comply with existing approval processes as applicable.  This guideline includes submitting to the governance council [if established] for review and approval all GenAI use cases that leverage foundational models, such as ChatGPT. However, use cases leveraging single-purpose models (e.g., translation models), or models that are built internally, may follow standard processes and principles, such as AI Governance[4] and Privacy by Design.
7. If you are working with a GenAI use case that has been approved through the governance process, you still must:
   • Exercise human oversight over all uses of GenAI.
   • Document how you used GenAI and any results you discarded, as well as why a specific output was used.
   • Communicate about the instances in which GenAI was used, including to the colleagues exposed to the output.
8. Turn off chat history if you plan to make any use of the chat.

[4] AI governance is a set of principles, policies, and practices that govern the development, use, and ethical implications of artificial intelligence (AI).