AI Project Management Hype

Introduction

It seems like everyone has jumped on the AI bandwagon proclaiming that AI will make huge improvements in not only automating routine tasks, but in deriving new insights by making never-before-seen correlations. There are different types of AI, but the one most talked about and being implemented today is Generative AI (GenAI) which uses algorithms and models to make predictions, decisions and generate outputs based on the input data.

And within GenAI, there are different types that vary based on task complexity and who does the work. Project Management Institute (PMI) is the leading authority on project management and they too have embraced GenAI and offer online courses to learn about it. In one of the courses, they provided a useful chart to understand the different types of GenAI that vary based on task complexity and who does the work.

As can be seen in the chart, the more complex the task, the more human intervention is required.

GenAI Automation

We have used some GenAI tools to automatically summarize meeting notes, and it’s useful to make sure that nothing was missed. The PMI course included a case study on analyzing documents (Lessons Learned) using ChatGPT. The Project Manager (PM) wanted to analyze lessons learned from hundreds of completed projects. The 1^st step was to consolidate the data, and the PM used ChatGPT to generate VBA code to do this after a number of iterations. The 2^nd step was to ask ChapGPT to identify the action items, but the PM was not getting any good action items until he converted the file into csf format.

I would have expected that ChatGPT could have used an unstructured format. More troubling was that the action items were all common-sense items that had already been implemented. The PM attributed the mediocre result to not including more lesson learned files (he limited it to a small number because of the work involved). He also thought if he had not sanitized the data (removing confidential customer and company information), ChatGPT would have done a better job.  So, the problem was the data and data need data governance.

Data Governance

Per the PMI course, data governance ensures the accuracy, completeness, and reliability of the data and includes:

• Data privacy – adhering to laws, regulations and standards such as GDPR, HIPAA or CCPA
• Data security – safeguarding access against unauthorized access
• Data lifecycle management – Overseeing the flow of data from creation to when it is deleted or archived
• Data architecture – Structuring the data so it can be used
• Risk management
• Data policies and procedures
• Data stewardship
• Data integration and interoperability
• Ethical use and accountability

That’s a lot of work to for any organization embarking on an AI project and seems to make sense for larger organizations with lots of data and resources. As well, by following all the guidelines, the GenAI results as in the case of the Lessons Learned example, are not worth the effort.

GenAI Risks

The course also warned about GenAI risks including:

• Biased data and unintended outputs
• Protecting sensitive data
• Using generated data without consent from the owner
• Lack of transparency of results and not being able to explain the reasons for it
• Hallucinations – can generate misleading outputs

But, in my opinion, the biggest risk, is that GenAI is making recommendations based on old data. In the example of analyzing lessons learned, the organization had already made the changes recommended. An organization must adapt to change, and the change may not be in the data being analyzed. As well, the solution to a problem, may require a creative approach not used before.

Reality vs Hype

We have all been blown away by ChatGPT giving us instance answers to any question, and we are now at what some would call the peak of inflated expectations. There’s a lot of work to go but Generative AI is rolling down the track and gathering momentum every day. There will be major breakthroughs in the not too distant future, and the reality will catch up with the hype.