Senior Data Engineer

Description

Role: Senior Data Engineer

Background:

Leveraging data analytics to provide insights and recommendations to drive strategic decision-making collaborating with cross-functional teams, including Finance, Accounting, Operations, HR, and others to deliver accurate and timely financial reporting, dashboards, analytics, and data-driven insights.

Key Accountabilities

A Senior Data Engineer (Production Support) will be responsible for monitoring, maintaining, and supporting ETL processes, data pipelines, and data warehouse environments. The ideal candidate should have strong troubleshooting skills, hands-on experience with ETL tools, and the ability to quickly resolve production issues to ensure data availability, accuracy, and reliability.

* Monitor and support daily ETL processes, data pipelines, and batch jobs to ensure timely and accurate data delivery.

* Troubleshoot and resolve production issues, job failures, and performance bottlenecks across ETL and data warehouse systems.

* Work Closely with Data platform team to resolve data load issues.

* Perform root cause analysis of recurring issues and implement permanent fixes.

* Collaborate with development teams to transition projects smoothly into production and ensure operational readiness.

* Implement and maintain monitoring, alerting, and logging solutions for proactive issue detection.

* Ensure data quality, consistency, and availability through ongoing validation and health checks.

* Apply best practices for production support, including incident management, change management, and problem management.

* Work closely with business users, data analysts, and other stakeholders to resolve data-related queries.

* Document runbooks, support procedures, and knowledge base articles to streamline production operations.

* Continuously optimize processes for reliability, performance, and scalability in production environments.

* Ensure compliance with data security, access controls, and audit requirements in production systems.

Day-to-Day Tasks - Senior Data Engineer (Production Support)

Production Support:

* Check system dashboards, logs, and alerts for failures or anomalies.

* Verify data quality and integrity checks (row counts, duplicates, missing data, schema changes).

* Review ETL/ELT job runs, data pipeline executions, and batch processes.

* Validate data loads into staging, warehouse, and downstream systems for critical tables.

* Monitor real-time and scheduled jobs to ensure SLAs are met.

* Investigate and resolve production issues (job failures, data inconsistencies, performance delays).

* Collaborate with business users to resolve data access or reporting issues.

* Coordinate with development/engineering teams for fixes, hot patches, or re-runs of failed jobs.

* Track and document incidents, resolutions, and preventive measures in ticketing systems (e.g., ServiceNow, Jira).

* Participate in daily/weekly operations meetings to report status and highlight issues.

* Handover critical ongoing issues to on-call/offshore support (if applicable).

Minor Works/ Maintenance:

* Enhance Existing models with addition of fields as per the requirements.

* Help with Deployments and initial loads during Go-live.

* Perform root cause analysis for recurring or high-severity incidents.

Proactive/Preventive Work:

* Fine-tune ETL workflows and SQL queries to improve performance.

* Implement monitoring scripts and automation to reduce manual intervention.

* Restructure the Load plans to improve effeciency.

* Review security and access controls to ensure compliance.

* Update documentation (runbooks, troubleshooting guides, SOPs) for operational continuity.

Skills and Capability requirements:

* 6+ years of experience with ETL, data pipelines, and data warehouse production environments.

* Strong expertise in troubleshooting ETL/ELT processes using tools such as Matillion, Informatica, ODI, or SSIS.

* Experience in cloud-based data platforms like Snowflake.

* Proven ability to analyze job failures, perform root cause analysis, and implement permanent fixes.

* Hands-on experience with monitoring, alerting, and logging tools.

* Familiarity with incidents, problem, and change management processes in ITIL-based environments.

* Strong SQL programming and debugging skills with relational and cloud databases.

* Experience with traditional and non-traditional forms of analytical data design (Kimbal, Inmon etc)

* Excellent communication skills to interact with business users, analysts, and cross-functional technical teams. xjnqpsq

Nice to Have

* Domain knowledge in the area of finance data is preferred.

* Experience with SAP Systems and Databases

* Knowledge of data visualization tools, such as PowerBI or Tableau

Ad ID: 5417890889