Difference between Extract, Transform and Load(ETL) and Enterprise Application Integration(EAI)

1. Overview

Although both Extract, Transform and Load(ETL) and Enterprise Application Integration(EAI) technologies seem surprisingly similar from an architectural view - where so-called adapters (or connectors) provide access to systems and data sources transformations take place to standardize proprietary formats, or routing capabilities are used to move packets of data - ETL and EAI serve fundamentally different purposes from an information management perspective.

ETL is typically used to move bulk data from application systems to data warehouses or data marts (all in a highly scheduled environment to avoid generating bottlenecks from high-volume transactions) while EAI is the technology of choice to connect systems for business process management and workflow.

2. Description

2.1 ETL

Extract, transform, and load (ETL) is a data pipeline used to collect data from various sources, transform the data according to business rules, and load it into a destination data store. The transformation work in ETL takes place in a specialized engine and often involves using staging tables to temporarily hold data as it is being transformed and ultimately loaded to its destination.

 

The data transformation that takes place usually involves various operations, such as filtering, sorting, aggregating, joining data, cleaning data, deduplicating, and validating data.

 

 

2.2 Enterprise Application Integration (EAI)

EAI, short for “enterprise application integration,” is a framework that makes use of software and computer systems to enable data integration across applications within a single organization, while also simplifying business processes among connected applications and data sources.

There are three main types of EAI:

• The Presentation Integration Model
• The Data Integration Model
• The Functional Integration Model

The sharing of data and business processes between applications is its primary purpose. However, EAI also defines a set of principles for the integration of multiple systems for communication architectures, such as message-oriented middleware (MOM).

EAI serves as a system that can provide a business service to simplify information data between diverse applications, which makes it possible to easily integrate them when needed.

This discipline of integrating applications and data within the enterprise has been a critical component of today's enterprise strategies. In fact, many vendors offer EAI suites that provide cross-platform, cross-language integration solutions.

2.3 Comparison of two concepts

Feature	ETL	EAI
Focus	Data Integration(Data warehousing)	Application Integration(Operational Apps)
Primary Technology	Database	Application
Timing	Batch	Real-time
Data	Historical	Transactional
Volume	Days or weeks of data(Records per min which is GB)	Single transactions(Messages/second which is KB)
Integration Initiation	Pull, query-driven	Push, pull, event-driven
Flow Control	Meta-data drove, complex data flow	Business-rule drove, workflow-oriented
Validation	Strong data profiling and cleansing capabilities	Limited data validations
Transactional	Limited transaction and messaging capabilities	Strong transaction control and recovery. Guaranteed message delivery with two-phase commit

2.4 When to use

When EAI	When ETL
A high number of transactions	Large amounts of data
Message transforms	Complex transformations
Transformation act on a single row of data	Transformation is data-set oriented
Little data augmentation	Large data augmentation
1 to n ; m to n	Point-to-point
Suitable for real-time data needs	Suitable for large amounts of data
High volume, low footprint data exchange	Generally used to move data between 2 or more databases/data repositories

3. References

https://en.wikipedia.org/wiki/Enterprise_application_integration

https://en.wikipedia.org/wiki/Extract,_transform,_load

http://www.ebizq.net/topics/eai/features/6595.html?&pp=1

https://docs.microsoft.com/ko-kr/azure/architecture/data-guide/relational-data/etl

http://www.itworld.co.kr/news/106925

https://www.mydbsync.com/blogs/eai-or-etl-when-you-should-choose-one-over-the-other/