AWS Data Engineer

From 3 to 6 year(s) of experience
₹ Not Disclosed by Recruiter

Job Description

Roles and Responsibilities

• Expertise in ETL optimization, designing, coding, and tuning big data processes using Apache Spark.
• Experience with building data pipelines and applications to stream and process datasets at low latencies.
• Show efficiency in handling data - tracking data lineage, ensuring data quality, and improving discoverability of data.
• Sound knowledge of distributed systems and data architecture (lambda)- design and implement batch and stream data processing pipelines, knows how to optimize the distribution, partitioning of high-level data structures.
• Experience designing and supporting large-scale distributed systems in a production environment


Desired Candidate Profile

• Should have strong experience in data engineering tools such as Amazon Redshift, Big Query.
• Should have strong working Experience open-source technology for data engineering such as Apache Spark, Airflow and stitch.
• Strong working Experience Apache Kafka for real time streaming data pipelines and applications.
• Good Working Knowledge on Hadoop cluster.
• Strong programming language in Python.
• Strong working experience on data models, data mining, and segmentation techniques.
• configuration management using Puppet, Ansible & chef Knowledge of Python, Java
• Expertise - Good hands-on knowledge of Source Code Management (Version Control System) tools like Git and Subversion.
• Proficient in developing Continuous Integration/ Delivery pipelines
• Experience with automation/ integration tools like Jenkins
• Working Experience of major cloud service providers, like AWS, etc.
• Good understanding of Infrastructure as Code (Programmable Infrastructure), and how to achieve that by using tools like Puppet, Chef, Ansible, etc.
• Hands-on knowledge of software containerization platforms like Docker and container orchestration tools like Kubernetes and Docker Swarm.
• Good Scripting experience in PowerShell/Bash/Python.
• Good Working Experience on REST API.
• Kubernetes implementations in major Cloud Providers (Amazon EKS).
• Experience in OpenShift and other providers will be a plus.



Role:Data Engineer

Salary: Not Disclosed by Recruiter

Industry:IT Services & Consulting

Functional Area:Engineering - Software & QA

Role Category:Software Development

Employment Type:Full Time, Permanent

Education

UG:B.Tech/B.E. in Electronics/Telecommunication, Computers

PG:M.Tech in Electronics/Telecommunication, Computers,MCA in Computers,MS/M.Sc(Science) in Computers

Company Profile

RS Consultants

A digital transformation company offering innovative IoT and custom-made SaaS solutions that enable organizations to embrace digitization holistically.
Company Info
X
View Contact Details+

Recruiter Name:Rahul

Contact Company:RS Consultants

Reference Id:DataEng0722

Website:http://jobs.rsconsulting.in