Core Generic/standard AWS scalable S3, Spark, Persist, Console

1. Microservices

1.1. adapters

1.2. Apis

1.3. Push or pull

2. Generic input Normalized

3. The idea is create adapter for each provider and generate a generic model, to be processed by the core

4. MIcroservices

4.1. Persist

4.2. Call processor

4.3. Dedup

4.4. xml importer

5. Generic output

6. Notes: Spring batch AWS- Data flow/pipeline ETL (EMR) S3 Spark Dinamo DB