The issue of data retention is becoming increasingly relevant and challenging for the many organisations out there currently focused on developing holistic data strategies. In particular, as the value of data in business and society continues to increase, so does the need to balance infrastructure and governance priorities.

The problem is, data retention is becoming a growing IT cost centre as businesses build the infrastructure required to retain and manage data volumes that are expanding at an exponential rate. However, this is not always conducive to effective compliance and regulation because an ineffective approach can increase levels of business risk, governance breaches and in some cases, lead to heavy fines.

As a result, there is considerable pressure on these critical organisational functions to implement effective retention strategies that meet both infrastructure and governance priorities. The difficulty is that these aren’t always compatible, resulting in a disconnect between the infrastructure and governance/privacy teams’ data retention plans.

While governance and privacy experts are in charge of developing policies that meet compliance, contractual or legal requirements, their infrastructure counterparts may lack the data visibility or tools needed to put them in place. What businesses need is a strategy based on an effective, integrated data retention strategy, but in practice, many are struggling to balance the two.

Infrastructure insight

Taking infrastructure first, data retention raises some basic issues for IT teams such as how can they find and apply the correct tools to provide the necessary levels of visibility and understanding necessary for good governance?

Existing technologies aren’t always up to the task of coping with increased policy complexity, putting IT strategy at odds with business requirements. In other cases, businesses succeed in implementing rules that meet legal and regulatory requirements, but this might come at the expense of users who are forced to work with inefficient, time-consuming processes.

This has the potential to have a negative impact on infrastructure and storage services. Data volumes, for example, may expand at an exponential rate to the point where organisations are retaining far more data than they require. Furthermore, all that increased data necessitates additional storage technology investments, increasing the cost, complexity and administration overhead encountered by infrastructure teams.

If that wasn’t enough to contend with, the industry-wide focus on retaining more data from more sources in general – an approach based on the premise that it will provide important benefits at some point in the future – exacerbates the issues.

Good governance

For their part, governance and privacy teams are also under pressure, in particular, to design rules that meet ever-tighter legislative and regulatory standards. In many situations, governance must be addressed even when it is technically difficult or incompatible with business procedures, adding to the difficulty that businesses face.

Indeed, some governance and privacy teams successfully develop the appropriate data retention policies to fulfil or exceed their requirements, but then struggle to put them into practice because they lack the necessary technology tools and technical expertise. The lack of insight into real-world data use is a particular problem, with organisations finding it impossible to reconcile policy aims with the needs of users who must implement on a daily basis.

So where does that leave us? To ensure data retention benefits infrastructure, privacy and governance alike, organisations must build synergy between policy and technology. This relies on the use of the appropriate toolkits so that teams can completely understand their data and, as a result, build rules that address business objectives and regulatory requirements. Ideally, this should be prioritised in tandem with the automation of retention regulations so that businesses can put data beyond use at the appropriate time.

This can deliver some important benefits, such as lowering the risk surface of the data organisations store and handle while also reducing the amount of data they maintain, thereby lowering technological and administration expenditure. It’s an approach that also allows teams to concentrate on complexity, because by keeping this under control, they can provide a variety of technical and operational benefits that help organisations remain nimble and compliant.

At a time when authorities have the power to levy massive fines for noncompliance, these are crucial considerations. As a result, organisations are increasingly focused on data retention plans that will keep them compliant without suffocating innovation, efficiency, or profitability. This is becoming a must-have for organisations focused on success in terms of maximising the value of data in the short and long term.