Last Updated on November 7, 2024 Sarah Gayda
Insights on unlocking the full potential of unstructured data through AI, process automation & strong records management practices from industry thought leaders
On the whole, data quality at organizations worldwide is quite poor. Meanwhile, estimates of the amount of unstructured data in organizations range from 80-90%.
This creates a very large problem for your company in several ways. First, your data is often unusable and not retrievable. Secondly, most of it is non-compliant and lacks proper governance. Thirdly, and perhaps most topical, if your data is unstructured and untagged, it is likely to create problems when you deploy Copilot and other AI applications.
A recent Colligo survey highlighted a similar trend. Just 19% of respondants rated their organization’s data quality as “Good – accurate, tagged, structured, with limited ROT”.
For more details, see our webinar Break Free – Overcome Poor Data Quality & Unruly Unstructured Data.
However, all is not lost.
Solutions for data quality problems from industry experts
One strategy overheard at the recent AIIM Information and Data Leadership Symposium (IDLS) is to define a threshold for what is deemed “clean.” For example, one of the panelists suggests aiming for 80%. In this case “clean” means structured, usable data that is not redundant, obsolete or trivial (ROT).
It is unlikely and unrealistic for most organizations to shoot for 100% data cleanliness. Instead, define what a win can look like at your organization. Then, aim to achieve cleanliness before running analytics on the data that informs business decisions.
At the same event, one expert said that data needs to be “findable, usable, and governed.” Otherwise, it is not valuable.
The retrievability of data and information is what makes it valuable to your organization. If you can’t find your data, its value is significantly diminished.
In many cases, the memory of your organization is saved in your unstructured data. So, it’s important to determine how to extract its value.
What about policies?
We’re advocates of strong records management policies, particularly around emails. However, where many policies fall short, is they are not easy for workers and users to execute on.
At Colligo, we aim to make policies very simple and easy to follow with our applications. We do this by bringing our apps right into the places workers are already spending their day (Outlook, Teams, Word, Excel, PowerPoint).
Still, policies remain a significant challenge for many organizations. Attendees at the AIIM IDLS event echoed this sentiment.
We believe that any policy needs to have a value proposition that should be articulated throughout your organization. Specifically, your value message needs to answer, “Why are we doing this?” and “What is the impact and benefit to my organization?”
For example, if you’re an insurance company, you may be susceptible to significant legal risk. Therefore, you may have much more stringent deletion and disposition rules than an unregulated, less sensitive or less litigated organization.
“Default delete”
“Default delete” is an approach where items are deleted after a certain period of time by default, unless otherwise specified. One way to do this is have a blanket delete policy (or the inverse of a retention policy). Then, provide users with a way to add a label or save to a separate location for items of value that need to be kept for a longer period.
While “default delete” is something that many in the records management world have been aspiring to for years, many organizations, still do the opposite and keep everything forever.
There are certain cases and documents that do need to be kept perpetually or for extended retention periods (i.e. 30 years) in Government and sensitive industries. However, the reality is that keeping less or only what your organization needs, would significantly improve data quality and cleanliness while reducing risk.
Colligo makes this very easy when it comes to emails and email records management. We make it simple to deem an email or attachment as a record in the process of saving it to SharePoint, and then adhere to the default deletion policies for documents not tagged as records.
Bridging the gap between the Records and AI teams
We’ve seen this evolve in several ways recently. We don’t know how this will play out in the longer-term, but we’re seeing a few discernable trends:
- Records management being re-organized into the data & analytics teams
- Records management reporting into the CIO directly instead of Legal
- Records management evolving into high-level Information Governance roles
All of these are positive trends. Our view is that the convergence of records, data and general information groups is a net positive. It allows your organization to consider all information in decision making. It also produces better results in terms of keeping information for the right amount of time, putting proper governance and classification in place, and allowing both records and AI team members to mobilize and take action.
This all feeds into the AI strategies that are emerging, as data governance is the foundation and pre-cursor to effective AI and Copilot deployments. For more on this, read Get Your Data AI Ready Now to Ensure Business Success or view our webinar: Is Your Data Ready for AI?
If you have questions about how to improve data quality or want more information about Colligo, get in touch.
