On-Prem to Cloud Data Migration via Informatica
Intertek achieved 100% SLA compliance, zero data loss, and seamless integration of five complex data sources with Cloudaeon.
What is Lorem Ipsum?
Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged.
Lorem Ipsum is simply dummy text of the printing and typesetting industry. Lorem Ipsum has been the industry's standard dummy text ever since the 1500s, when an unknown printer took a galley of type and scrambled it to make a type specimen book. It has survived not only five centuries, but also the leap into electronic typesetting, remaining essentially unchanged.
A leading global player in testing, inspection, and certification services, Intertek is a 130-year-old company. Headquartered in London, it operates in more than 100 countries with 44,000+ workforce heading towards a common goal and contributing excellence to a wide range of industries, including healthcare, construction, food, and automotive.
As every modern organisation wants to move to the cloud, so did Intertek aim to. Collaborating with Cloudaeon to move from on-premise databases to the cloud. Intertek wanted to modernise its data infrastructure by migrating key business data from on-premise databases to the cloud.
The aim was to enable scalable analytics, enhance data accessibility, and lay the foundation for a future-ready data platform.
Challenges
Complex multi-region data sources
Managing data migration across five disparate on-prem databases (4 SQL in Singapore, 1 Oracle in Houston) was a chaotic situation. These databases were difficult to maintain and had multiple access protocols and latency issues.
Informatica-limited environment
Considering the data volume and complexity, a robust system was required. However, due to an existing 3-year subscription with Informatica, it was a constraint rather than using potentially more agile tools.
Data sensitivity and masking requirements
Intertek deals with a lot of sensitive data that needs to be handled at the field level. Masking that data on a real-time basis before storage in the cloud to comply with data governance and privacy policies.
Schema and pipeline changes on the fly
The existing platform needed frequent schema changes and modifications which disrupted the ongoing ETL jobs. It further caused downtimes affecting the end-to-end process.
Weekend maintenance and support windows
Regular patching and RDC updates had to be scheduled during the weekends. This needed tight coordination and timely execution to prevent delays in weekday analytics.
Solutions
End-to-end ETL pipeline development using Informatica
Built and maintained ETL pipelines using Informatica Intelligent Cloud Services (IICS) to extract and load data from multiple on-premise databases. To ensure high data quality and availability, incremental loads with full-load jobs and CDC pipelines were implemented.
Secured data transfer via virtual machines and secure agents
To securely connect the on-prem data sources with the cloud destination, Informatica Secure Agents on dedicated virtual machines were deployed. This helped maintain compliance with enterprise security policies.
Data privacy through masking pipelines
To ensure compliance with data governance standards, Cloudaeon developed custom data masking pipelines within Informatica to handle sensitive fields before cloud ingestion.
Dynamic scheduling and monitoring
Informatica’s scheduler and trigger features were used to configure job schedules based on business frequency and load requirements. These jobs were further monitored through Informatica logs accessed via a Jumpbox (bastion host), resulting in quick diagnosis and recovery from failures.
Agile change management and weekend support
To ensure smooth data flows, Cloudaeon provided ongoing schema change support by incorporating new fields and modifying existing pipeline logic. Additionally, conducted patching and RDC updates to minimise downtime and maintain operational continuity.
Tool evaluation and advisory
While Informatica was ultimately used due to a multi-year license agreement, Cloudaeon also evaluated alternative tools like Azure Data Factory (ADF) and Talend, offering advisory input to ensure future scalability and cost-efficiency.
Impact
Achieved 100% response and resolution against SLA consistently.
100% Data availability with no data loss or service-impacting issues across all ingestion sources.
Successfully onboarded and integrated five complex data sources (SMART, GSCC, GSCC Audit, iCare GTS, Interlink) into stable BAU support.
