How to succeed in a data-driven digital transformation

In recent years, driven in part by the adoption of cloud technologies, data has taken center stage as a vital currency in determining the companies who succeed in the digital age. As a result, companies today are transforming their data ecosystems and operations as a critical step in reimagining their businesses for the digital world.     

The main goal of these digital transformations is to effectively leverage modern technologies, such as artificial intelligence and machine learning, for data insights that can help better serve customers. With businesses making fewer than 50% of their decisions based on quantitative information, instead relying on “gut feeling, experience or opinion,” getting clear insights from data can be difficult. And with global spending on digital transformations to reach $1.8 trillion in 2022, an 18% increase from the previous year, it’s more important than ever that companies get this undertaking right. 

Whether a company is in the beginning stages of planning such wide-reaching digital change or has already begun the journey, it’s helpful to understand what a digital transformation truly means.  

While technical facets of digital transformation, like data migration, are vital, our own 10-year transformation has shown us that lasting enterprise-wide change that results in real business value must also include a new approach to data management and a cultural shift around data that empowers lines of business.  

Digital transformation is the ongoing process of integrating new technologies, culture and procedures around data into an organization to increase its competitiveness and better serve customers. Every digital interaction provides an opportunity to personalize or improve experiences, so it’s no surprise that data is at the heart of many digital transformations today.                                                   

Another key goal of a data-driven digital transformation is to create an environment in which it is easy for everyone to consume and produce data.

Digital transformation is a journey rather than a destination to reach. Companies that succeed have built a foundation for continuous innovation and growth as new data technologies emerge. Other benefits may include:

• Insights-based decision-making that leads to performance gains
• Improved employee productivity and greater operational efficiency
• More engaging and personalized customer experiences

Although what works for one company will differ from another, a key step includes modernizing existing data technologies and infrastructure. This can include overhauling a legacy on-premise system for the cloud, building a data streaming platform to make real-time insights possible or moving data to an enterprise-wide data lake.  

So what do companies that have undergone a successful data-driven digital transformation look like? They often display some or all of the following characteristics:

• Data delivery in real-time for faster insights
• Treatment of data as a product
• Self-service capabilities with built-in governance
• Federation of data ownership to business lines 
• Centralized policy and tooling to manage data warehouse, data lake and low latency data stores

Data migration involves the moving of data from one system to another. This can involve a move between locations, formats, applications or all of the above. Organizations most often initiate a data migration because they’ve outgrown the legacy platform and are looking to replace it with a new system while keeping the same dataset. Many times the new platform offers important benefits such as better costs, security or performance. In a data migration, there is a target destination and end date. The goal is to reinstate what was in the old system in the new system while making as minimal changes as possible in the data’s structure and processes.

While seemingly straightforward, data migrations can be quite challenging and 83% of them fail or exceed their budgets and schedules. They also involve a great deal of planning and follow-up activities following the migration. Challenges can arise from data quality issues or a lack of standardization across data sets that slow the process of moving data into the new system. Despite the difficulties companies may encounter, data migration is a crucial and necessary step for businesses today, particularly as more companies embrace the benefits of cloud technology. 

The main steps in a data migration include:

• Data conversion
• Data profiling
• Data cleansing
• Data validation
• Quality assurance
• Data movement

Today, many data migrations are part of a wider transformation strategy involving the move from an on-premise data infrastructure to the seemingly unlimited storage and scalable compute resources of the cloud. More than 70% of companies have migrated some part of their workloads to the cloud, according to Gartner.

While data migration is an important aspect of many digital journeys, it is not in itself a digital transformation. A complete transformation must go beyond technology and consider change in the culture, process, accountability and governance of surrounding data.

Additionally, while a migration has a target destination and well-established steps to getting there, a data transformation does not end and will look different for every business. 

Transformations are only sustainable and successful when accompanied by a lasting cultural change; cultivating behaviors and attitudes that support the company’s new direction and objectives. A study by BCG found that 70% of digital transformations fail to meet their objectives. A sticking point for many of these failed initiatives was the “people dimension,” or the operating model and culture needed to advance a company’s digital plan. This can involve changes in organizational structure or associates’ skills. Successfully implementing new ways of working with data across organizations establishes a cycle of continuous improvement for enterprises.

Some companies may find they do not need to plan for an entire transformation in order to fulfill their objectives. For example, many smaller companies are by nature much more agile than large enterprises and can make changes quickly without the need to overhaul their legacy systems or processes. Additionally, some companies are forced to plan for and implement a data migration because of a looming deadline, such as space running out in their on-premise system.

But when a business can decide the timing of each, tackling data migration and wider transformation separately is usually preferred. While data migration has the convenience of a target and timeline, comprehensive transformation takes much longer and requires more resources. Companies must also leave themselves room to learn and grow from the mistakes that are inevitable in a transformation journey, which should not be tied to implementing a successful data migration.

While each transformation journey may look different from one company to the next, the following are some of the best practices emerging as more companies become data-driven in their transformation strategies.

A comprehensive data strategy can act as a roadmap and guide for all stakeholders in a broader transformation. The strategy should not be created in a silo but should involve key members from across the organization. It should include an overall business objective to the transformation as well as goals for individual business lines. The strategy should also be set up in a way where it can adapt and change over time to new business conditions and disruptive technologies. 

Focus on strong communication across all levels of the organization as well as the establishment of a feedback loop. Bring together all key stakeholders, including data engineers, data consumers, managers of lines of business, executives, before executing any data strategy. Discuss the goals, frustrations and wants from each stakeholder and also establish key dates to meet.

A true transformation cannot succeed without the buy-in of your people. Each leader is responsible for making a case to each end user on how their lives will be better in the new world. Understandably, change is difficult and can lead to fears around the significance of current roles and skepticism about whether the change is a good thing. By clearly and thoughtfully communicating the value of the transformation for each employee and how a role will change, if at all, businesses can cultivate a supportive and empowering culture for data end users.

A true digital transformation requires changing the culture of an organization to uphold the adoption of new organizational structures or operational processes for the foreseeable future. 

During our own 10-year data journey, we found a change in the way we viewed and used data across the organization was a necessity and not an option. A significant cultural shift was in coming to treat data as a product across the organization where data end users are consumers and data sets are delivered as products with their own SLAs. 

Moving to a decentralized approach where data ownership was assigned to each line of business helped us address the significant increase in data complexity and volume that resulted from our move to the cloud. The model of a central data team managing data requests from across the organization became a bottleneck and simply could no longer support the new data ecosystem and the demand for faster insights.

Many companies are recognizing the need to invest in data as a crucial part of their digital transformation journeys. For most of us, the challenge is in recognizing that deeper, long-lasting change is about more than just the data itself, but rather the way your organizational culture complements and enables new ways to produce, consume and govern data.

Companies must establish the data strategies, best practices and cultural shifts that will sustain transformation efforts well beyond the current environment. By approaching each step with the full scope of technological and organizational changes needed in mind, businesses will be set up for continuous innovations for the benefit of their customers.