Microsoft’s conversational AI technologies
Microsoft is a platform provider and developer of computer software, hardware, and related products. One AI technology it has developed and deployed is conversational AI.
Conversational AI ranges from FAQ chatbots to virtual assistants who can assist users to complete tasks or help them make decisions based on data analysis. For example, someone might ask ‘why are sales in this city lower than others?’.
Microsoft provides the building blocks for conversational AI technology through tools like the Microsoft Bot Framework. Microsoft’s partners and customers then customise and deploy that technology across the intended applications where people will use the technology. Microsoft supports its partners and customers to responsibly use the conversational AI technologies it provides to them.
This case study on conversational AI is based on Microsoft’s experiences as both a developer and deployer across various applications. For the pilot, Microsoft:
- focused on conversational AI ethical impacts, as this technology is widely used here and internationally
- compared its internal ethical processes and advice to customers with the Australian AI Ethics Principles.
Microsoft’s existing ethical processes
Microsoft has found it is an ongoing journey to instil a company-wide culture and approach to operationalising responsible AI. It has a range of multi-disciplinary committees, working groups and champions including:
- an AI, Ethics and Effects in Engineering and Research (Aether) Committee with representatives from engineering, consulting, legal and research teams who:
- advise the company’s leadership on responsible AI issues
- review sensitive applications
- drive related innovations on responsible AI
- an Office of Responsible AI which sets company-wide rules and advocates for responsible AI externally
- responsible AI Champions who communicate and align responsible AI practices across teams.
While Microsoft has established new governance mechanisms and processes, this may not work for other organisations. There is no one-size-fits-all approach. For example in financial services, Microsoft is aware some banks have augmented existing operational or ICT risks frameworks to consider AI ethics issues. Smaller organisations may also find it easier to change existing processes rather than establish new ones.
Applying the AI ethics principles to conversational AI
Microsoft’s starting point in applying the Australian AI Ethics Principles was to ask ‘Are the Australian AI Ethics Principles even relevant to conversational AI?’ On face value, people might assume conversational AI does not result in significant ethical impacts on people – and in many situations this is likely to be the case.
Potential impacts of using AI technologies should always be considered from the beginning, even where the AI application appears benign. Asking initial questions can help developers and deployers to focus on the most relevant ethical considerations. For example:
- Will the AI system be used in ways that significantly impact others?
- Will the AI system impact on human rights (such as the right to privacy, and the right not to be discriminated against)?
For example, customer service chatbots are a common conversational AI technology unlikely to cause severe harm. However if a business uses or manages chatbots inappropriately, it may cause negative impacts, such as:
- personal data leaks involving customer details and purchase history
- data breaches or other risks due to inappropriate security controls
- discrimination or ‘hate speech’, for example when bots engage in ‘conversation’ on topics unrelated to customer service
Working through the Australian AI Ethics Principles assists organisations to make informed choices about how to avoid ethical impacts. Examples of broader design or deployment decisions include:
- whether a chatbot should ‘talk’ or engage in various languages
- what processes should be available for customers to be handed over from a chatbot to a human
Decisions on how to design and deploy a chatbot so that it meets the Australian AI Ethics Principles will depend on the context. For example, handovers from a chatbot to a human are important where:
- AI informs advice on government emergency services
- there are no alternative providers and an immediate solution is critical.
By contrast, handovers may be less important where customers encounter problems using AI to order takeaway food. Clearly this may still cause customer frustration, but that AI application has less significant impact compared to the government emergency example.
Putting AI ethics principles into practice
Microsoft’s core principles for responsible AI align with the Australian AI Ethics Principles. The company shared the following observations on applying them to conversational AI.
Principle 1: Human, social and environmental wellbeing
A chatbot’s purpose needs to be clearly defined. The team should assess whether:
- the purpose is beneficial
- it can design the bot to perform responsibly, or if not
- it can make changes needed for the bot to perform responsibly.
Principle 2: Human-centred values
Businesses should inform their customers at the start of an interaction that they are not interacting with a human. This ensures respect for users and prevents them feeling as though their autonomy has been infringed. Where there are potentially significant negative impacts, the chatbot should be able to redirect users to a human when requested. These ‘human hand-off mechanisms’ give humans control over bot functions and respect individual preferences for engaging with virtual chatbots.
Businesses should seek diverse views on an AI system’s benefits and risks early in its development and throughout its lifecycle. If users with a disability aren’t consulted for example, the team may overlook issues like designing chatbots in line with accessibility standards.
- Ask users whether they had a negative experience at the end of an interaction with a chatbot.
- Include functions for a human decision maker to act on or override chatbot recommendations or outputs.
- Communicate appropriately so as not to pretend the bot can do more than it can.
See Microsoft’s Guidelines for Human-AI Interaction.
Principle 3: Fairness
One way businesses can detect and mitigate bias is by considering input data and algorithm design, and putting audit systems in place. Microsoft provides an open-source toolkit to assist developers assess model fairness and mitigate negative impacts of bias (visit Microsoft’s FairLearn toolkit website). Businesses also need strategies to address bias and implement supporting governance processes.
Businesses also need to consider the potential for discrimination.
Useful practices to prevent chatbot and user interactions resulting in discrimination:
- Limit topics a bot will cover so that it steers clear of any sensitive topics.
- Apply machine learning and keyword filtering to enable bots to detect and respond appropriately to sensitive or offensive input from users.
- Use other available tools to reduce risks of discriminatory outcomes (for example, the Fairlearn toolkit).
- Clearly define the intended purpose of using a bot, and grounding design choices in that purpose.
- Design bots to address potential discrimination from technical and social lenses.
Lessons from Microsoft’s Tay experience
Microsoft’s own experience with Tay – a chatbot it deployed in 2016 – generated important lessons. Microsoft learnt how conversational AI systems may feed off positive and negative interactions with people. Malicious or ignorant users can train an AI-powered chatbot to exhibit negative behaviours.
Chatbot design needs to address discrimination technically and socially. The social lens needs to anticipate the wide ranging interactions society will have with the chatbot.
Making conversational AI accessible
Chatbots can provide new opportunities for people with disabilities to access government services, interact with businesses, and work more efficiently.
To unlock this potential, businesses can make inclusive design choices allowing people with disabilities to more easily interact with chatbots. For example, a design team can build chatbots with voice activation capability to improve access for visually impaired persons. Procedures for referring customers back to humans also need to be accessible for people with disabilities.
- Have user-friendly chatbot interactions by voice (this gives people with disabilities more suitable options to interact and fill out user profiles).
- Design your chatbot to be inclusive to a diverse workforce by including multiple languages and formats.
- Adjust bot interaction with different local accents.
- Build in functionality to support multiple language requests for digital assistants likely to engage an international audience.
See Microsoft’s inclusive design toolkit.
Making conversational AI more inclusive
British Broadcasting Corporation (BCC) partnered with Microsoft to build an end-to-end, customised digital voice assistant. BBC had to ensure its chatbot was multilingual to include its diverse workforce and audience or customers. It adjusted bot interactions with different local accents.
BBC also prioritised retaining trust with chatbot users through secure privacy controls. It worked with Microsoft to maintain complete control over the data use. This meant BBC could use data to improve services, while still maintaining transparency for users on how the data was used.
Principle 4: Privacy protection and security
Chatbots can learn a great deal about its users, so privacy considerations are especially important. Businesses must obtain user consent to collect and process data. Businesses should keep users informed about what and how their data is used.
All Microsoft’s development teams must follow a privacy standard. Teams take steps to maintain privacy, through practices like data privacy impact assessments.
- Implement user controls like an easy-to-find ‘Show me all you know about me’ button, or a profile page for users to manage privacy settings.
- Include additional rights to users where a bot is sharing data, such as allowing the user to:
- choose the data sharing recipients
- choose what information is shared
- opt out where possible.
- Refer to available technical methods for ensuring privacy in an AI context.
- Address cybersecurity considerations. Microsoft applies its Security Development Lifecycle as a standard for AI products.
- Incorporate additional or specific mitigations where necessary. AI security mitigation is an ongoing area of research. For example organisations should build capability to identify when an interaction is going ‘off script’ in ways that suggest a malicious attack. Such an interaction could be from another bot, being used maliciously to trigger specific responses from the target chatbot.
- Set systems to continually monitor for security vulnerabilities such as:
- internal data left unprotected during system updates
- accidental changes to firewall setting.
Principle 5: Reliability and safety
Businesses should ensure AI systems operate as intended and that they identify and address performance outside expected parameters. To improve the accuracy of conversational AI, businesses may set clear metrics and closely monitor performance data for any issues. These key metrics can include:
- acceptable error rates in a bot
- desirable ratio of positive to negative interactions.
The relevant error and reliability thresholds will vary according to the use case. A bot used for army recruitment should have a lower margin for error than one used to purchase socks. However, the ideal error rate for any use case should be one closest to zero.
- Use technical tools to track a model’s functioning and lineage, and detect errors over time. See Microsoft’s Azure MLOps and the open source InterpretML toolkit.
- Enable a chatbot to explain how users can fix an issue themselves if the bot can’t handle the request. For example, re-phrase a question or hand off to a human operator.
- Regularly update the bot’s dialogue models to ensure constant improvement and a better customer experience.
Principle 6: Transparency and explainability
For Microsoft, being transparent means being able to provide meaningful and relevant information to stakeholders in different contexts. In Microsoft’s experience with conversational AI, both technical and non-technical transparency are necessary.
For example, Microsoft has developed non-technical transparency measures that highlight the ethical AI issues that can occur.
Businesses need to consider how they are transparent when making choices about:
- notifying users that they are interacting with a virtual bot
- including accessibility features and multi-language choices
- providing short user surveys to get feedback on reliability.
Principle 7: Contestability
Microsoft considers contestability especially relevant when the chatbot has the potential to have a significant impact on individuals, society or the environment. Even for less sensitive applications, Microsoft considers it important to have mechanisms in place for humans to be brought into the loop if needed.
Microsoft uses practices such as:
- allowing users to easily identify how and where to make a complaint or raise a concern
- notifying users upfront about available options to make complaints and seek redress
- advising users where to find information on how decisions are made.
Businesses should tailor these practices to the use case depending on its sensitivity. For example, where a chatbot is designed to recommend retail purchases, it is unlikely a human needs to review this decision beforehand, unless a user requests a human assessment or has further questions.
Principle 8: Accountability
Microsoft wants to see AI developers and deployers ask appropriate questions and ensure they use the AI system responsibly. All these businesses have complementary responsibilities on ethical impacts throughout an AI system’s lifecycle. As a developer, Microsoft ensures it understands:
- how Microsoft itself, as well as other buyers, will use the technology
- potential applications
- potential impacts stemming from applications.
Those deploying the AI systems should consider how the AI technology might result in negative impacts based on their intended use.
If a business designs and develops a bot for a third party to deploy, both parties need to have a shared understanding. They need to know who is responsible for what, and document that understanding .
Microsoft assists its partners and customers on responsible approaches by:
- providing clear technical and non-technical tools and information
- educating them on the software’s capabilities and limits, and best practices for avoiding negative impacts.
Benefits and impacts
Through this pilot, Microsoft has learnt:
- implementing ethical AI is an ongoing journey
- each AI application reveals new insights and lessons for the future
- internal governance, procedures and engaging with external stakeholders is critical to effectively embedding ethical AI.
Microsoft strongly supports the government’s initiative to pilot the Australian AI Ethics Principles:
‘Translating AI principles into practice requires asking important questions and making complex choices.
‘We encourage further pilots between government agencies, industry and other stakeholders. These pilots allow for collaboration in identifying issues which can then be turned into improved tools for responsible AI.
‘We encourage the government to explore additional pilots with regulators or bodies, particularly in financial services or healthcare where AI is increasingly used.
‘Microsoft is committed to responsible AI in how we design, develop and deploy AI systems. Our goal is to make sure our AI systems are fair, reliable, safe and trustworthy.’
See Microsoft’s responsible AI resources
If you have any questions about this example or are interested to learn about Microsoft's ethical processes, see Microsoft’s responsible AI resources site.
Learn more with Microsoft’s: