What Should Compliance Look for in AI-Generated Research Notes?

As Artificial Intelligence (AI) tools like ChatGPT and Google Gemini become increasingly integrated into family office workflows, compliance teams face new challenges and responsibilities. These AI-powered tools offer powerful capabilities—generating draft research notes, summarizing vast reports, and assisting with preliminary due diligence. Yet, their outputs come with risks, notably hallucinations—convincing errors that can mislead decision-makers and ultimately expose the organisation to regulatory and reputational harm.

For family offices managing ultra-high-net-worth portfolios, where private markets and scarce data are common, vigilance around AI-generated content is imperative. This blog post explores what compliance teams should look for when reviewing AI-generated research notes, with a focus on audit trails, source documentation, and ensuring factual integrity.

Why AI Hallucinations are a Compliance Concern

“Hallucinations” in AI refer to fabricated or incorrect information produced by language models that sound plausible or authoritative. Both ChatGPT and Google Gemini excel at crafting human-like text, but they do not inherently verify facts. Their outputs depend on patterns learned from training data rather than live databases or authoritative sources.

For compliance teams, the risks posed by hallucinations include:

• Misleading investment decisions: AI-generated notes that contain subtle inaccuracies may influence investment committees or portfolio managers without proper scrutiny.
• Regulatory non-compliance: Inaccurate research records can violate documentation requirements and audits.
• Impaired risk assessment: Erroneous facts may skew risk profiles, especially in opaque private markets.

It’s essential not to treat AI outputs as fully reliable sources but rather as starting points that require validation.

Common Family Office Use Cases for AI-Generated Research

Understanding typical family office use cases for AI helps frame compliance review criteria. Key scenarios include:

• Preliminary research: Quickly scanning background information on assets, sectors, or managers before deeper human due diligence.
• Parsing lengthy reports: Summarizing complex manager reports, market commentary, or regulatory filings.
• Creating executive summaries: Condensing multiple data points for higher-level stakeholders.

In each case, AI tools like ChatGPT or Google Gemini can accelerate workflows but risk introducing errors if not critically assessed.

Private Markets: Scarcity Increases Risks

Family offices often allocate substantial capital to private equity, venture capital, real estate, and other illiquid or bespoke investments. These areas generally suffer from limited data availability and transparency compared to public markets. AI models trained predominantly on public web content or historical texts may lack up-to-date or complete information in these domains.

This scarcity creates a heightened likelihood of hallucinations or omissions in AI-generated notes about private market opportunities, valuations, or counterparties.

Examples of Risk in Private Markets

Scenario Risk Compliance Review Focus AI summary of niche private equity fund Fabricated track record metrics or team bios Verify against fund documents or third-party data providers Parsing a private real estate report Incorrect valuation assumptions or market trends Cross-check with appraisals and market research Early-stage venture capital analysis Misstated product status or competitive landscape Confirm with direct manager contacts and recent filings

Faithfulness vs Factuality: Understanding the Differences

When evaluating AI outputs, compliance should distinguish between two crucial concepts:

• Faithfulness: The AI's ability to keep the generated text consistent with the input data or source documents provided to it.
• Factuality: The truthful accuracy of the content with respect to reality or verified facts independent of the input.

For example, an AI summarizing a 50-page manager report is expected to be faithful—meaning it accurately reflects what’s in that report. However, the underlying report itself might contain outdated or incorrect information, so factuality cannot be inferred solely from faithfulness. Moreover, hallucinations occur when the AI invents information that is neither in the input nor factual.

Compliance teams must evaluate both:

1. Does the AI-generated note correctly reflect the source materials? (faithfulness)
2. Do those source materials and the resulting summary hold up to verification? (factuality)

Key Compliance Review Checklist for AI-Generated Research Notes

The following checklist serves as a practical guide for compliance professionals auditing AI-generated research content.

1. Source Documentation & Audit Trail

• Track the input: Ensure every AI output links back to clearly identified source documents (reports, data extracts, transcripts).
• Retain original materials: Store source reports and data alongside the AI-generated notes in compliance repositories.
• Maintain logs: Record tool usage metadata—date, user, prompt text, AI version (e.g., ChatGPT’s model iteration, Google Gemini release).

2. Verification & Fact-Checking

• Cross-reference facts: Validate critical data points against authoritative sources such as manager filings, independent databases, or direct communications.
• Watch for hallucinations: Flag any statements that appear overly confident, vague, or unsupported.
• Prioritize private markets scrutiny: Apply additional diligence to opaque investment areas.

canadianfamilyoffices.com

3. Language and Presentation Review

• Check for overconfidence: AI outputs may assert facts without hedging; compliance should ensure appropriate disclaimers or uncertainty language.
• Recognize vague claims: Identify any general statements like “uses the latest AI model” without detail and challenge them.
• Restrict usage as input: Prevent AI outputs from being treated as definitive or cited sources without corroboration.

4. Governance and Policy Controls

• Define AI usage guidelines: Establish when and how AI tools can be used in research workflows.
• Train staff: Educate analysts and compliance officers on AI limitations and review responsibilities.
• Monitor compliance reviews: Conduct periodic audits documenting adherence to controls.

What Would We Show an Auditor?

One of my guiding questions when assessing any new technology, including AI tools, is: “What would we show an auditor?” For AI-generated research notes, the answer should include:

• A complete audit trail linking inputs, prompts, and AI outputs.
• Documented evidence of source material verification.
• Records of reviewer checklists and any flagged issues or corrections.
• Policies governing acceptable AI use and user training logs.

If any of these elements are missing, your compliance framework is incomplete—risking penalties and internal control failures.

Conclusion

AI tools like ChatGPT and Google Gemini offer transformative potential for family offices by automating and streamlining research and due diligence processes. But their compelling language can conceal hallucinations—plausible yet inaccurate information that poses significant compliance risks.

Compliance teams must adopt rigorous review protocols emphasizing audit trails, source documentation, and thorough fact-checking, especially in the private markets context. Differentiating faithfulness to input documents from factual accuracy in the broader context is essential.

By asking “What would we show an auditor?” before adopting and relying on AI-generated notes, family offices can harness these technologies responsibly, maintaining the integrity and trust central to ultra-high-net-worth portfolio stewardship.

References & Further Reading

• OpenAI ChatGPT Official Blog
• Google Gemini Announcement
• “The Risks of AI Hallucinations in Investing” – CFA Institute