This role is responsible for the development and implementation of data warehouse solutions for company-wide application and managing large sets of structured and unstructured data. The candidate will be expected to analyze complex customer requirements and work with data warehouse architect to gather, define and document data transformation rules and its implementation. Data Engineers will be expected to have a broad understanding of the data acquisition and integration space and be able to weigh the pros and cons of different architectures and approaches
Translate the business requirements into technical requirements
ETL development using native and 3rd party tools and utilities
Write and optimize complex SQL and shell scripts
Design and develop code, scripts, and data pipelines that leverage structured and unstructured data
Data ingestion pipelines and ETL processing, including low-latency data acquisition and stream processing
Design and develop processes/procedures for integration of data warehouse solutions in an operative IT environment.
Monitoring performance and advising any necessary configurations & infrastructure changes.
Readiness to travel to customer sites for short, medium or long-term engagements.
Create and maintain technical documentation that is required in supporting solutions.