There is one challenge that all insurance companies have in common – the bottleneck of getting data from a raw to refined state for the purposes of analytics and predictive modelling. The goal is to collect data, prepare it, transform it, and ultimately standardise it as quickly and efficiently as possible so that it can be leveraged by insurance companies in a competitive way.
To achieve that, insurers must engage in a process called data wrangling. Also referred to as data preparation and data munging, this process enables companies to transform and map data from one ‘raw’ data form into another format in order to make it more compatible and valuable for downstream purposes like analytics.
But some insurers are stuck in the past, using manual processes like Excel spreadsheets and other text or code-driven platforms to manage data for risk insights and claims analysis. Those old-fashioned data wrangling programs are not equipped to deal with the size or complexity of the data that insurers have access to today.
According to executives at Trifacta, a data preparation SaaS adopted by some of the largest insurance companies worldwide, if insurers rely on outdated tools and strategies for data management, this will negatively impact their performance. In order to improve their workflows, a proper data preparation tool is vital.
“We’ve approached a situation now where the ecosystem has evolved in such a way that people are trying to do more with less. While that’s always the goal, insurers are doing it now in what we would call the modern stack ecosystem, which includes the cloud-based platforms,” said Chris Moore (pictured), director for North American sales engineering and solutions at Trifacta.
Many property and casualty (P&C) insurers today are using weather data in order to better calculate and predict where they should provide coverage for property risk. Those weather data sets are new, complex, and they come in very large volumes that require sophisticated data preparation and wrangling tools.
“The tools that insurers have been using for the last 20-years are generally not good at keeping up with that amount of volume,” said Moore, “and they’re often more limited in terms of enabling people to understand the data they’re working with and shape it into the insights they want at scale.”
Data preparation is key across the entire insurance value chain. With the right tools for claims analysis, for example, insurers can accelerate common tasks that are required for predictive modelling or loss forecasting, and turn them into more automated, more predictable pipeline workflows. The insights provided via data wrangling can also help to uncover patterns, which are helpful in determining risk appetite, desirable market capacity, and even insurance fraud.
“It has become a universal known across everybody in insurance and really any data-heavy industry that prepping the data is almost always where the real heavy lifting is done,” Moore told Insurance Business. “There's a tremendous amount of time spent in just getting data prepared, cleaned and ready for it to be leveraged by the analytical teams, model development teams and other value-added activities in the organisation. We're happy to be the data janitors for those folks, because we know that’s where a lot of the time is spent before any of those valuable insights can ever really be gleaned.
“Insurers have access to such a diverse range of datasets, and the talent to process all of that data is spread all over. Not everybody has the amount of money to hire all the quants and the data scientists in the world, and at the same time, the talent that they have is needing to be skilled up to learn how to prepare data with new tools, on new platforms, and with new data sets that they've never seen before. We’re there to help them make that challenge easier without everyone needing to be a qualified data scientist.”
