Senior Data Engineer (Full Stack) (CPT/JHB/PTA)
IT – Analyst, Data Management, IT – Software Development
Cape Town – Western Cape, Johannesburg – Gauteng, Pretoria/Centurion – Gauteng
ENVIRONMENT:
YOUR thorough understanding of best practice business intelligence implementations as a Senior Data Engineer (Full Stack) is sought to play a crucial role in driving the business forward of a cutting-edge provider of End-to-End Technology & Services. You will work together with the best-of-breed Solution Architects to design and implement state-of-the-art cloud, hybrid and on-prem solutions while wrangling (Big) data from multiple sources into a reliable asset while designing and implementing the best possible data pipelines for each data source and data store, as well as deliver impactful advanced analytics using ML, AI, Cognitive Services, and Data Science. You will require a Bachelor’s Degree in IT, Engineering or other Sciences with 6+ years of work experience in a similar role.
DUTIES:
Delivery –
- Wrangling (Big) data from multiple sources into a reliable asset that is aimed at evolution instead of maintaining a status quo.
- Work together with the best-of-breed Solution Architects to design and implement state-of-the-art cloud, hybrid and on-prem solutions.
- Design and implement the best possible data pipelines for each data source and data store, as well as deliver impactful advanced analytics using ML, AI, Cognitive Services, and Data Science.
- Work along the full spectrum of descriptive, diagnostic, predictive and prescriptive analytics.
- Familiarise yourself with ML algorithms and AI approaches like clustering, regression, classification, speech recognition, OpenAI, LLMs, vision, search, and translator.
- Understand DataOps and MLOps approaches to ensure solutions will continuously be deployed in short delivery cycles with maximum business impact and the highest quality standards possible.
- Ensure data lineage and all metadata is maintained effectively.
- Design and develop the sexiest possible dashboards for business.
- Use your combination of technical and data analytics expertise to improve existing or proposed business innovations.
People –
- Work with multi-skilled teams that contain experts from neighbouring crafts like Software Engineering, User Experience and Quality Assurance.
- Demonstrate maturity and soft skills like time management and motivation.
- Influence change within your team.
- Build relationships and collaborate within your team.
- Work with teams to solve complex business problems, and make an impact, while pushing yourself and others to become more.
- Coordinate with teams to meet quality standards and timelines.
- Mentor and grow others.
- Hold others accountable to a standard of excellence.
- Galvanise team members.
Customer –
- Being pragmatic in making the best technological choices for the customer’s needs.
- Identify opportunities for service improvements and strategic alignment.
- Provide insights to business owners to validate their ideas and guide them to make the most informed decisions possible with the most desirable outcome.
- Being prudent when it comes to data governance. We want to ensure that our customers can fully trust the data assets that we build out for them.
- Flex your story-telling muscle when assembling the presentation layer for business owners.
REQUIREMENTS:
Qualifications –
- A Bachelor’s Degree in IT, Engineering or other Sciences.
Experience/Skills –
- Minimum 6+ years of experience working as a Data Engineer within an enterprise environment.
- Experience working in a Delivery team together with other disciplines to deliver a product.
- Strong ability to model and analyse data.
- Solid understanding of Data Engineering methodologies.
- Good understanding of visualisation best practices.
- A desire to design and implement modern advanced analytics solutions and modern data warehousing solutions.
Company Tech Stack –
Cloud environments: Azure, AWS, Google
Front-end: Power BI, Microsoft Fabric, Amazon QuickSight, Google Data Studio, Grafana
Languages: SQL, Python, Scala
Data processing: Azure Data Factory, Amazon Glue, Apache Airflow, Databricks, Spark SQL
Relational databases: MSSQL, PostgreSQL, Oracle, MySQL, Azure SQL, Amazon RDS
NoSQL databases: MongoDB, Cosmos DB, DynamoDB, Neo4j
Streaming: Azure Stream Analytics, Kafka, Databricks streaming, Spark streaming, Google Dataflow, Amazon Kinesis
ML Platforms: Azure Machine Learning, Databricks ML, Amazon SageMaker, Dataiku
AI: Azure OpenAI, Azure Cognitive Services, AWS Cognitive Services, Google Cognitive Services
ATTRIBUTES:
- Strong attention to detail but viewed from a ‘bigger picture’ perspective.
- To collaborate and bring your unique contributions to a team environment.
- An ambition to continually improve your skills, embrace challenges and be driven by excellence.
