AI assistants at work and confidential data

Is it safe to use AI chatbots at work with confidential information? That depends on the tool and on what is entered, because when people type or paste information into an AI assistant, that information goes to a service run by someone else. AI assistants can draft emails, summarise documents and answer questions, and this guide explains the risks and what the Office of the Australian Information Commissioner (OAIC) and the Australian Signals Directorate (ASD) say about them.

Why it matters

The ASD’s guidance Engaging with artificial intelligence notes that “Information given to these systems may be incorporated into the system’s training data”, where it could shape outputs for users outside the organisation. It says organisations “should be cautious with what information they, and their personnel, provide” to generative AI systems.

The OAIC explains that some commercial AI products “include terms or settings that allow the product owner to collect the data input” by customers “for further training and development of AI technologies.” It also warns that once information is entered, it will be “very difficult to track or control how it is used, and potentially impossible to remove the information from the system.”

What the OAIC says about public AI tools

The OAIC’s guidance states: “As a matter of best practice, the OAIC recommends that organisations do not enter personal information, and particularly sensitive information, into publicly available generative AI tools, due to the significant and complex privacy risks involved.”

It contrasts this with proprietary systems, noting that an organisation using “a proprietary AI system rather than a publicly available chatbot” can check whether it “has protections in place to ensure that information entered into the system will not be disclosed outside the organisation.” It also says deploying a system locally or on premises “is likely to be more privacy-preserving”.

The ASD’s guidance refers to training staff “on what data can and cannot be input to the system”. Neither source publishes a single list for every organisation; what counts as confidential depends on the organisation and its obligations.

What to check before choosing an AI tool

The OAIC says organisations should not adopt AI products “simply because they are available”, and describes due diligence that “should not amount to a ‘set and forget’ approach”. Questions and considerations it raises include:

• “What data has the product been trained and tested on?”
• “Will the developer or other third parties have access to the data which your organisation inputs or generates”
• “Are there any known or likely security risks?”
• where the servers are located, and “whether personal information could be disclosed outside of Australia”
• whether features that would disclose personal information can be turned off.

The ASD’s guidance also describes:

• requiring “phishing-resistant multi-factor authentication” for access to AI systems (see Microsoft 365 MFA methods compared)
• granting privileges “based on the need-to-know principle and the principle of least privilege” (see Microsoft 365 admin roles and least privilege)
• logging and monitoring the system’s inputs and outputs
• seeking vendors “that are transparent about how they have developed and tested their AI systems.”

Workplace assistants connected to your data

Some workplace assistants work with an organisation’s own files and email. Microsoft’s documentation says “Microsoft 365 Copilot only accesses data that individual users have at least View permissions to access within Microsoft 365 services like SharePoint, OneDrive, and Teams.” Microsoft’s SharePoint guidance adds that “Copilot and agents retrieve data from Microsoft Graph and respect existing permissions, sharing settings, and policies”, and suggests adjusting SharePoint and OneDrive sharing settings “to help prevent accidental oversharing in Copilot and agentic experiences.” See SharePoint oversharing and Copilot.

Privacy obligations still apply

The OAIC states: “Privacy obligations will apply to any personal information input into an AI system, as well as the output data generated by AI (where it contains personal information).” Its guidance also says:

• businesses “should update their privacy policies and notifications with clear and transparent information about their use of AI”
• public-facing AI tools, such as chatbots, should be “clearly identified as such” to external users
• “AI systems are known to produce inaccurate or false results”, so outputs need checking.

If an organisation is covered by the Privacy Act, the OAIC’s guidance is the authoritative source. This article is general information, not legal advice.

How to set rules and training for staff

The OAIC’s checklist asks: “Do you have policies in place that outline the permitted and prohibited uses of the system?” It also asks whether staff have been trained “about the operation of the AI system”, and whether there are processes for ongoing monitoring and regular audits or reviews.

The ASD’s guidance describes training staff on how far outputs can be relied on, and setting up organisational processes for checking AI outputs. Our guide to AI governance basics covers writing a simple AI use policy, and security policies for small teams covers policies more broadly.

Common questions

Is it safe to use AI chatbots at work?

It depends on the tool, its settings and what is entered. The OAIC’s best-practice position is that organisations don’t enter personal information, and particularly sensitive information, into publicly available generative AI tools. The ASD notes that information given to these systems may be incorporated into their training data. See what the OAIC says about public AI tools, or using AI chatbots: privacy, accuracy and scams for personal use outside work.

Can I put client or personal information into a public AI chatbot?

The OAIC recommends, as a matter of best practice, that organisations do not enter personal information, and particularly sensitive information, into publicly available generative AI tools, citing significant and complex privacy risks. It also states that privacy obligations apply to any personal information input into an AI system. What else counts as confidential depends on the organisation and its obligations. See privacy obligations still apply.

Do AI chatbots use what I type to train their models?

Some do. The OAIC explains that some commercial AI products include terms or settings that allow the product owner to collect customers’ inputs for further training and development. The ASD similarly notes that information given to generative AI systems may be incorporated into training data, where it could shape outputs for other users. The OAIC suggests checking whether such features can be turned off. See what to check before choosing an AI tool.

Is a business version of an AI tool different from the free one?

It can be. The OAIC notes that an organisation using a proprietary AI system, rather than a publicly available chatbot, can check whether it has protections to ensure entered information will not be disclosed outside the organisation. It also says deploying a system locally or on premises is likely to be more privacy-preserving. The tool’s own documentation and terms say what applies to each version.

What do I do if confidential information has already been entered into an AI tool?

The OAIC warns that once information is entered, it may be very difficult to track or control how it is used, and potentially impossible to remove it from the system. The tool’s terms and settings describe whether inputs are used for training. Privacy obligations apply to personal information input into an AI system, and for organisations covered by the Privacy Act, the OAIC’s guidance is the authoritative source.

Can Microsoft 365 Copilot see all our company files?

According to Microsoft, it reaches only what each user can already view. Microsoft says Microsoft 365 Copilot only accesses data that individual users have at least View permissions to in services like SharePoint, OneDrive and Teams, and that it respects existing sharing settings. Microsoft suggests adjusting SharePoint and OneDrive sharing settings to help prevent accidental oversharing in Copilot. See workplace assistants connected to your data and our Copilot oversharing guide.

Where to get official help

The OAIC’s guidance on privacy and commercially available AI products is the authoritative source on Privacy Act obligations. The ASD’s Engaging with artificial intelligence covers security.

Related guides

• Prompt injection explained, for defenders
• SharePoint oversharing and Copilot
• AI agents and permissions