Plot of the described mechanisms on the Big Data Blueprint

Data governance activities and application functions layers for maintaining qualities of the datasets

Application functions and activities for securing datasets and storage functionalities

--

Usage of the processed data in work processes in the consumer organisation

Visualisation of the data or information for interpretation by analists and users

Analysing the cleaned, aggregated and filtered datasets

Storage of primary and temporary datasets for usage in higher layers of the architecture

Processing activities for filtering, cleaning and transformation of datasets

Selecting data sources on information content and data qualities

Not part of a data architecture but the input of the architecture, it is the raw material of a data pipe and therefore an essential part

Big data sets can be voluminous so compressing has advantages for storage and/or processing

Processing of data for transformation, cleaning and filtering of data

Administration and registration of security concepts in a Big Data environment.

Serialization and deserialization of proprietary datasets to standardized stored datasets

Portal functionality to give users and administrator access to the other logical functionalities in a Big Data environment.

Transformation and transfer functionality from data sources to storage, processing and usage.

Functionality for analysing the transformed and standardized data in a data analytics tool.

Coordination of automated tasks for example in batch processes and mapreduce.

Visualisation of query and analytics results to data scientists and end users.

Functionality to design workflows to automatically perform transformations and cleaning of datasets.

Management of clusters in a distributed or cloud based infrastructure

Retrieval and querying function on the transformed and standardized datasets

Management function for the various resources in a Big Data environment

Storage function of big data sets.

Register function for the governance of data sets in relation to consumers, business owners and qualities.