Absa: Big Data Engineer – Senior

Individuals within the Big Data Engineer role ensure that data pipelines are scalable, repeatable, and secure, and can serve multiple users within the Organization. They help facilitate getting data from a variety of different sources, getting it in the right formats, assuring that it adhere to data quality standards, and assuring that downstream users can get that data quickly. This role usually functions as a core member of an agile team.

Key Responsibilities

• Translates business requirements into system requirements.
• Selects data solution software and defines hardware requirements.
• Works with IS team members to design and implement the data solutions in alignment with the project schedule.
• Creating strategies that use business intelligence and data platforms.
• Works with the IS team members to design and implement data solutions in alignment with the project schedule.
• Builds and architect next-generation Big Data analytics framework developed on a group of core technology.
• Conducts logical and physical database design.
• Designs key and indexing schemes and designs partitioning.
• Develops, tests, implements, and maintains database management applications.
• Analyse data across multiple systems and multiple formats and provide analytics and information that provide integrated views necessary to assist with business concerns
• Develop and analyse analytical data sets across multiple sources of data, creating meaningful visualisations and design and implement creative approaches to predictive modelling problems
• Design, develop and run model assessments, validation, enhancement activities and balancing procedures to ensure integrity of data
• Leads and communicates across third parties, technical, and business product managers on solution design.
• Conducts demonstrations, briefings and status reports as required.
• Performs technology and product research to identify opportunities that impact business strategy, business requirements and performance.
• Evaluates and provides feedback on future technologies and new releases/upgrades.
• Provides technical assistance to junior team members and to colleagues across the Institute.

Competencies:

• Deciding and initiating action
• Learning and researching
• Relating and networking
• Adapting and responding to change
• Persuading and influencing
• Creating and innovating

General

• The appointment will be made in line with the Divisional Employment Equity strategy

Essential/Basic Qualifications

• Bachelor’s in computer science, computer engineering, or equivalent work experience
• Typically requires over 5 years data engineering, data science, or software engineering experience.
• Requires 3-5 years of demonstrated experience leading teams of engineers.
• Capability to architect highly scalable distributed systems, using different open source tools.
• Demonstrated experience with agile or other rapid application development methods.
• Demonstrated experience with object oriented design, coding and testing patterns as well as experience in engineering (commercial or open source) software platforms and largescale data infrastructures.
• Understands how algorithms work and have experience building high performance algorithms.
• Extensive knowledge in different programming or scripting languages.
• Expert knowledge of data modelling and understanding of different data structures and their benefits and limitations under particular use cases.
• Experience using Big Data batch and streaming tools.
• No Criminal Record

Preferred Qualifications

• Preference will be given to South African Citizens and Permanent residents of South Africa in possession of proof of permanent resident status