Sr. Data Engineer (AWS Tech Stack)
Location: Pune (WFO)
Experience: 7+ Years
Job Description
We are seeking a highly skilled Data Integration Architect to join our team. The ideal candidate will have extensive experience with the AWS tech stack and a strong background in data integration, ETL processes, and cloud architecture. As a Data Integration Architect, you will be responsible for
designing, implementing, and managing data integration solutions to support our organization\'s data strategy.
Roles & Responsibilities
∙Architect and Design:
∙Design and implement data integration solutions using AWS services.
∙Develop architecture blueprints and detailed documentation.
∙Ensure solutions are scalable, secure, and aligned with best practices.
∙Implementation of Logs, Alerting and Monitoring at various stages of Data flow.
∙Data Management:
∙Manage and optimize ETL processes to ensure efficient data flow.
∙Integrate data from various sources into data lakes, data warehouses, and other storage
solutions.
∙Ensure data quality, consistency, and integrity across the data lifecycle.
∙Data Pipeline Management – Ensuring the data ingestions and pipelines are robust and scalable for huge data sets.
∙Collaboration and Leadership:
∙Work closely with data engineers, data scientists, and other stakeholders to understand data requirements and deliver solutions.
∙Lead technical discussions and provide guidance to the development team.
∙Conduct code reviews and ensure adherence to coding standards.
∙Performance Optimization:
∙Monitor and optimize the performance of data integration processes.
∙Troubleshoot and resolve issues related to data integration and ETL pipelines.
∙Implement and manage data security and compliance measures.
∙Semantic and Aggregation layer design and review
∙Dashboard performance tuning (Looker)
Qualifications:
Must have
∙Education and Experience:
∙Bachelor’s or Master’s degree in Computer Science, Information Technology, or related field.
∙Minimum of 7 years of experience in data integration and ETL processes.
∙Extensive experience with AWS services such as AWS Glue, AWS Lambda, Amazon Redshift, Amazon S3, and AWS Data Pipeline.
Technical Skills:
∙Proficiency in SQL and experience with relational databases (e.g., MySQL, PostgreSQL).
∙Strong programming skills in Python
∙Experience with data modeling, data warehousing, and big data technologies.
∙Familiarity with data governance and data security best practices.
Soft Skills:
∙Excellent problem-solving and analytical skills.
∙Strong communication and collaboration abilities.
∙Ability to work independently and as part of a team.
Good to Have
∙AWS Certified Solutions Architect or equivalent certification.
∙Experience with machine learning and AI technologies.
∙Familiarity with other cloud platforms (Google Cloud).
∙Experience and exposure to US health care and Life science
∙GCP Look ML
