Roles and Responsibilities
We are hiring an AWS Senior Data Engineer
Location: South Africa (Onsite)
- Expertise in ETL optimization, designing, coding, and tuning big data processes using Apache Spark.
- Experience with building data pipelines and applications to stream and process datasets at low latencies.
- Show efficiency in handling data - tracking data lineage, ensuring data quality, and improving discoverability of data.
- Sound knowledge of distributed systems and data architecture (lambda)- design and implement batch and stream data processing pipelines, knows how to optimize the distribution, partitioning of high-level data structures.
- Experience designing and supporting large-scale distributed systems in a production environment
Desired Candidate Profile
- Should have strong experience in data engineering tools such as Amazon Redshift, Big Query.
- Should have strong working Experience open-source technology for data engineering such as Apache Spark, Airflow and stitch.
- Strong working Experience Apache Kafka for real time streaming data pipelines and applications.
- Good Working Knowledge on Hadoop cluster.
- Strong programming language in Python.
- Strong working experience on data models, data mining, and segmentation techniques.
Salary: Not Disclosed by Recruiter
Industry:IT Services & Consulting
Functional Area:Engineering - Software & QA
Role Category:Software Development
Employment Type:Full Time, Permanent
UG:B.Tech/B.E. in Electronics/Telecommunication, Computers
PG:MS/M.Sc(Science) in Computers,MCA in Computers,M.Tech in Computers