Summary of use case presentation:

In this presentation, I demonstrated how artificial intelligence (AI) can be leveraged to enhance and streamline the peer review process for practicing accounting and auditing firms. By integrating a firm's quality control policies and procedures, peer review guidelines from the Institute of Chartered Accountants of India (ICAI), and other relevant knowledge into an AI system, many aspects of the peer review process can be automated and made more efficient.

I showcased two key applications of AI for peer review:

1. An AI-powered tool to assist in filling out the peer review application in accordance with the firm's quality control manual and ICAI guidelines. By training the AI on these knowledge bases, it can provide guidance, suggestions, and validations as the application is being completed, improving accuracy and consistency.
2. A chatbot that can field common questions about the peer review process from practitioners. The chatbot draws upon the same knowledge repositories to provide instant, relevant answers and point users to appropriate resources. This can save significant time for reviewers and support staff.

Use Case Video - https://www.youtube.com/watch?v=_skggONmLw0

Introduction

Peer review is a critical quality control mechanism in the accounting and auditing profession. It involves an independent evaluation of a firm's quality control system and its compliance with professional standards, regulatory requirements, and the firm's own policies and procedures. The objective of peer review is to promote quality in the accounting and auditing services provided by firms and to protect the public interest by enhancing the reliability and credibility of financial reporting. However, the peer review process can be time-consuming, complex, and resource-intensive, especially for smaller firms. This is where artificial intelligence (AI) can play a transformative role in streamlining and automating key aspects of the peer review process.

Tools used in the presentation:

1. Language model AI, ChatGPT 4o (https://chatgpt.com) to ingest the ICAI peer review materials, firm quality control manual, and other knowledge bases.
2. About Language model AI, ChatGPT 4o (https://chatgpt.com): ChatGPT is a large language model trained by OpenAI, capable of understanding and generating human-like text. It was used to process and analyze the ICAI peer review materials, firm quality control manual, and other relevant knowledge bases to create a structured dataset for training the AI-powered peer review application and chatbot.
3. Chatbot platform, botpress (www.botpress.com) to design and train the interactive agent.
4. About Chatbot platform, botpress (www.botpress.com): Botpress is an open-source conversational AI platform that enables developers to build and deploy chatbots across various channels. It provides a user-friendly interface for designing conversational flows, integrating with NLP services, and managing the chatbot lifecycle. Botpress was used to create the interactive chatbot for handling practitioner queries related to the peer review process.

The key steps to follow the use case were:

For creating GPT on Peer Review Process

1. Gather and preprocess all the source material to create a clean, structured knowledge base. Say in my case I have gather various Peer Review publications released by ICAI including the Peer Review Guidelines, Peer Review Checklist etc.
2. Login to the www.chatgpt.com using the Gmail credentials.
3. Ensure the Version being used is paid version which enables one to create their own GPT.
4. I have created my own GPT named “Simulated Quality Control Reviewer” wherein I have given the instructions containing the skills, expectations and relevant information for this GPT to perform Peer Review and also to help the users in completing the Peer Review application.
5. With the help of GPT’s “Configure” option I have ensured that the GPT is configured as per the expectations of a Peer Reviewer as well as that of a Practicing Unit.

For creating Chatbot:

1. Visit the website www.botpress.com while logging in using the Gmail credentials.
2. Click on the “New bot” and then click on “Open in Studio”
3. In the next screen click on “Start from Scratch” and then click the option “Use Template”
4. In the following screen a flow diagram with option Start, Add_stuff_here (Node) and End are visible. Select the middle part i.e., Node and delete the same while confirming the deletion
5. After deleting the node, Right-click on the screen and selection the option “Standard Node” and then click on “Add Card” provided on the Standard1 node.
6. In the pop-up window, to the left of the screen, select “Raw input”. On the Right side of the window, against the “Raw input” enter the default question which will be displayed to the user. In my case I have mentioned “What is your query w.r.t GST”
7. Then select the option “Store result in” and under Select/create variable type the random name for Query input. The name I have given is “query1”
8. On the left side of the screen, select the option “Default Knowledge Base” and upload the document which is intended to be used for responding to the queries through Chatbot. In my case I have used the GST Legal Updates provided by the GST committee of the ICAI.
9. After adding the knowledge base, on the Right side of the screen select the Default Knowledge base under the title “Included Knowledge Bases”
10. Again Right-click on the screen and selection the option “Standard Node” and then click on “Add Card” provided on the Standard1 node and select the option called “AI Generate Text”
11. For this card i.e., “AI Generate Text”, using the Right side of the screen under Prompt field type @ and select the input query field i.e., “@query1”
12. Under Output variable type the random name for the Query output. The name I have given is “outputquery”
13. On the same node, below the “AI Generate Text” I have added another card and selected the option “Text” for the said card. On the Right side of the screen for this “Text” card under the field “Message to send” type @ again and this time select the output query field name i.e., “@outputquery”
14. That’s it, on the top of the screen click the option “Publish” which will deploy the chatbot and provides the link that can be shared with the users by using which the users can chat with the Chatbot on the GST both from the knowledge base and also using OpenAI chat.

Importance and Impact of the this presentation:

The peer review process is critical for maintaining high standards of quality and ethics in the accounting and auditing profession. However, it can be time-consuming and complex, especially for smaller firms with limited resources. By automating key aspects of the process and providing intelligent support tools, AI has the potential to greatly increase efficiency, reduce errors and inconsistencies, and allow reviewers to focus their expertise on higher-level analysis and decision making. Firms can complete the review process faster and with greater confidence, while freeing up staff for other billable work. Over time, AI can also identify patterns and insights from aggregated peer review data to surface areas for industry-wide improvement.

Long-term benefits of AI in peer review:

Beyond immediate efficiency gains, AI can drive long-term improvements in the peer review process and the accounting and auditing profession as a whole. As more firms adopt AI-powered tools and chatbots, the aggregate data generated from peer reviews can be analyzed using machine learning techniques to identify patterns, trends, and best practices. This can help surface areas for industry-wide improvement, such as common deficiencies, emerging risks, or opportunities for enhancing professional standards and guidance. AI can also facilitate knowledge sharing and benchmarking across firms, enabling them to learn from each other and continuously raise the bar for quality and compliance.

Alternative AI Tools:

Other potential tools for building AI-assisted peer review applications include:

1. IBM Watson
2. Microsoft Azure Cognitive Services
3. AWS AI services
4. Google Cloud AI
5. No-code AI platforms like Levity AI

The choice of tool may depend on the firm's existing technology stack, data security requirements, budget, and in-house AI skills. However, the fundamental architecture - using a large language model to encode domain knowledge and power user-facing interfaces - would likely be consistent across different toolchains.

Summary

The use case presentation demonstrates the immense potential of AI in transforming the peer review process in the accounting and auditing profession. By leveraging language models and chatbot platforms, firms can automate and streamline key aspects of the peer review workflow, reducing manual effort, enhancing consistency, and enabling reviewers to focus on higher-value activities. As AI technologies continue to evolve, there is scope for further integration with other audit tools, data analytics platforms, and regulatory reporting systems, creating a more seamless and intelligent peer review ecosystem. Firms that proactively embrace AI powered solutions for peer review are likely to gain a competitive edge, improve the quality of their services, and strengthen public trust in the profession. The future of peer review lies in the synergy of human expertise and artificial intelligence, working together to uphold the highest standards of integrity and excellence in accounting and auditing.