Data Management

Data Lifecycle Management

The Zetaly data platform supports data lifecycle management from Release 5.0.0.
This support is available regardless of collection method.
Data lifecycle management is an approach to managing data throughout its lifecycle, from collection to deletion.
This is done by breaking down the data journey into processes, then automating each of them, enabling data to flow conditionally from one stage to the next, benefiting from controlled and personalised enrichment.

Data lifecycle management has three main objectives:

• Ensure data integrity.
• Guarantee data availability.
• Securing data.

The data lifecycle can be categorised into 5 stages:

• Creation and collection.
• Storage and management.
• Use and sharing.
• Archiving.
• Deletion.
  
  All these objectives and stages are carried out by different modules of the Zetaly data platform.

Zetaly Data management

Zetaly data is managed by defining simple management rules.
These rules are defined on the basis of the company's data management policy.  (when to perform an action on my data and what type of action).
A rule defines when (Older Than) and what type of action (Aggregate, Delete) must be performed on the data. This applies regardless of the nature of the data.
Only the periods and actions relating to the data collected must be taken into consideration. (see Managing rule for more information).

• When do I want to carry out an operation on my data?
• What type of action I want to perform on my data ?

The rules defined constitute a common core repository.
From this repository, you can use the available rules to create rule sets. These rule sets represent and constitute complete data life cycles.
They are the management strategies that can be applied to the data. 

A set of rules is called a “strategy” in the Zetaly Data Platform. 
This definition establishes the characteristics of a complete data lifecycle, which can be associated with one or more sets of collected data. 
The rules can be reused multiple times to create as many strategies as necessary to comply with the company's policy.
(see Managing Strategie for more information).



 
Any data collected in the Zetaly data platform has an associated strategy.
This enables data lifecycle management to be standardised according to the characteristics of the data itself and the company's retention policies.

A same strategy can be applied to several Datasets, but each Dataset is associated with a single strategy.

The strategy applied by default is called ‘Orphan’. It does not perform any action on the data.
To apply a data lifecycle management policy, you need to assign another strategy to the dataset.

When a policy is assigned to a dataset, the associated rules are automatically applied. The lifecycle is defined.

Once the lifecycle has been defined, it is necessary to specify the aggregation operations that will be performed on the dataset and any fields that need to be indexed.
Standard SQL functions are available according to the field's data type to define the required aggregation profile for the dataset.
When creating the aggregation profile, it is possible to specify several operations (e.g. MIN, AVG, MAX) on the same field in order to enrich the content of the dataset.
In this case, new Dataset fields are automatically created for each additional operation declared.

Optionally, during each stage of the lifecycle (rules) you can choose not to retain fields in order to optimise storage space and/or not to keep data that is no longer of value to the business.
Before updating Dataset fields profile a strategy must be applied to the Dataset.

Once all the mandatory definitions have been created, the strategy can be activated at dataset level.

On activation, storage objects are created and the associated rules activated. The lifecycle is applied to the data collected for this Dataset.

Each storage object contains the data resulting from the execution of the associated rule. An API is also automatically created for each storage object to access the data.

Rules run automatically without intervention, according to their own characteristics.
Statistics interfaces enable you to track activity and monitor the execution of lifecycles by Dataset.

The lifecycle applied to the dataset can be deactivated at any time. In this case, aggregation is no longer performed. However, data continues to be collected according to the configuration of collectors and connectors.
A deactivated strategy can be reactivated to restart lifecycle management.
For datasets with an inactive policy, you can unassign it. This allows you to simply delete the data or create an archive that will remain accessible for future use.