As info flows between applications and processes, it requires to be gathered from a number of sources, relocated across networks and consolidated in one place for refinement. The process of gathering, transporting and processing the data is called a virtual data canal. It generally starts with ingesting data right from a supply (for example, database updates). Then it moves to its destination, which may be a data warehouse just for reporting and analytics or an advanced info lake with regards to predictive stats or equipment learning. Along the way, it goes through a series of modification and processing actions, which can incorporate aggregation, blocking, splitting, merging, deduplication and data duplication.

A typical pipe will also have got metadata associated with the data, that can be used to keep track of where that came from and how it was refined. This can be used for auditing, security and compliance purposes. Finally, the pipe may be providing data being a service to other users, which is often referred to as the “data as a service” model.

IBM’s family of evaluation data control solutions incorporates Virtual Data Pipeline, which provides application-centric, SLA-driven automation to increase application expansion and screening by decoupling the control click this link now of test duplicate data via storage, network and hardware infrastructure. It does this by creating online copies of production data to use designed for development and tests, when reducing you a chance to provision and refresh those data replications, which can be about 30TB in proportion. The solution as well provides a self-service interface designed for provisioning and reclaiming electronic data.