The data integration tool market has established a focus on transformational technologies and approaches demanded by data and analytics leaders. The presence of legacy, resilient systems and innovation all in the market together requires robust, consistent delivery of highly developed practices.

The market for data integration tools includes vendors that offer software products that enable the construction and implementation of data access and data delivery infrastructure for a variety of data integration scenarios. These include Data acquisition for business intelligence (BI), analytics and data warehousing — Extracting data from operational systems, transforming and merging that data, and delivering it to integrated data structures for analytics purposes. The variety of data and context for analytics is expanding as emergent environments — such as non-relational and Hadoop distributions for supporting discovery, predictive modeling, in-memory DBMSs, logical data warehouse architectures and end-user capability to integrate data (as part of data preparation) — increasingly become part of the information infrastructure. Data integration challengers are intensifying because of the increased demand to integrate machine data and support Internet of Things (IoT) and digital business ecosystem needs for analytical processes.

Data integration tools provide the ability to ensure database-level consistency across applications, both on an internal and an interenterprise basis (for example, involving data structures for SaaS applications or cloud-resident data sources), and in a bidirectional or unidirectional manner. The IoT is specifically exerting influence and pressure here. Data consistency has become critical with new functionality in DBMS offerings — hinting that the battle for data integration is heating up to include the traditional data management vendors.

The explosion of data because of social media and the ubiquitous use of mobile devices has increased the need for data lakes or Hadoop-based DWs. The need to integrate non-relational structures and distributing computing workloads to parallelized processes (such as in Hadoop and alternative NoSQL repositories) elevates data integration challenges. At the same time, it also provides opportunities to assist in the application of schemas at data read time, if needed, and to deliver data to business users, processes or applications, or to use data iteratively. In addition, the differing structure of IoT or machine data is introducing new integration needs.

Intelligencia has a long history of integrating DI solutions, such as SAS DI, Datameer, HANA, Hadoop, amongst others. We also spend considerable time and effort integrating data streaming solutions like Spark, Flink, Storm, and Streambase, with standard EDWs to increase customer experience technology.