Created on 2024-10-10 21:28
Published on 2024-10-11 12:51
Today, I'm introducing a very simple application to demonstrate one way AI tools can speed up and enhance existing workflows. This tool accepts PDF files of audit reports and uses AI tools like OpenAI's GPT-4o-mini or Claude Sonnet 3.5 to do two things: 1) extract consistent, useful information from the audit report to create a structured summary, and 2) build on the audit results to suggest potential future work based on the summary. You can also download the results to a .txt file.
I built this application last week to show a few impressive capabilities that I believe will be important to consider when applying AI tools to workflows. The link is at the end of the post if you want to try it out.
For context, I've been a legislative auditor in the past, and it was common for our staff to review and summarize every single internal audit report published by a government agency, university, or even certain contractors. The goal was to read through the audit report, record information about the audit objectives and findings, and then determine if there were significant issues that indicated we (the legislative auditor) should perform additional work on the program or agency.
Basically, the task was to review audit reports, summarize the relevant parts, and use judgment to decide if the report should be used in planning future audits. In the manual process, the quality of summarizing, identifying significant issues, and generating potential new audits varied depending on the experience level of the auditors performing the task. An experienced auditor with a broad range of experience could do a great job. But for a new person assigned to this role, it was often hit or miss whether they could identify useful information to include in our process.
The goal of this tool is to show that a process that was done manually with inconsistent results can be redesigned using AI tools to be performed quickly, consistently, reliably, and customized as needed to plug into downstream processes.
The first notable capability I'm demonstrating with this tool is using AI tools to extract consistent, structured information from unstructured text data that can be stored or used in additional workflows.
Structured data, like tables in spreadsheets, is easy to manipulate, summarize, or extract for whatever task is at hand. Unstructured data, like text in PDF audit reports, is much harder to work with in existing workflows. PDFs often contain customized formatting, styles, graphs, figures, and tables that are not easily extracted by most automation tools. Audit reports have consistent types of information included in the text but where and how that information is presented can be different for each organization publishing audit reports.
The second capability is using the AI-extracted summary to suggest additional audit ideas for the future audit plans. With this step, we leverage the extracted summary and the AI tool's general knowledge to generate new audit ideas based on the summary.
Hopefully, you'll get the chance to try the tool. I wanted to show a simple example of how AI tools can enhance existing workflows by speeding them up, outputting consistent results, and building on the output to generate actionable ideas for humans to use. I would love to hear your thoughts if you try it out. If you have any questions about how it works or how I used Replit to create the tool, let me know!
https://auditanalyzer.replit.app/
Note: This tool was designed to be used with publicly available audit reports. I am not saving any pdfs or queries used with this tool, however, the text from PDFs is sent to OpenAI and Claude for analysis and the output is logged by Replit, the tool I used to build the app.