6 month Contractor required to assist in maintaining and extending the Data Platform to meet the needs and requirements of the Business stakeholders.
A very hands-on specialist technical role to support and improve AXA Insurance UK data systems under the guidance of team leads and the data solution architects. As a member of Data Services you will be supporting and enhancing the current Big data and Data Warehouse environment. The team member will be collaborating with other Data Services teams and other business as well as IT teams. As a Big data developer with ETL skills, should be able to ingest and process data on Hadoop as well as provide data feeds where required. You will be the go-to person within the team for Python development, Spark, Hadoop ecosystem, Cloudera specific tools like Cloudera Navigator and Cloudera Manager, T-SQL and SSIS. Main Duties & Responsibilities
Develop and support the Cloudera Hadoop implementations in Data Services.
Good knowledge of testing in a Big data environment.
Developing, testing and implementing components for the existing data solutions using industry best practice techniques.
Excellent coding and deployment practice.
To work as part of a team assisting and supporting other team members.
To provide support and assistance to other members and departments of the business as needed. This includes, but is not limited to software developers, underwriting, software testing, marketing and management.
Troubleshooting, debugging and improving existing processes.
Produce technical and user documentation for processes and procedures.
Skills, knowledge and behaviour
Essential skills and knowledge
Proficient in Python and SPARK development
Well experienced working on Hadoop environments with various suite of services [Oozie, HIVE, Impala, ELK, Kafka, Storm, Sqoop etc]
Experience working with various formats – XML, JSON,AVRO, PARQUET etc.
Strong experience in Data Ingestion and processing on Big Data
ETL and Data Warehousing using Microsoft BI Stack
Strong self-sufficiency and initiative working on Business Intelligence projects
Performance tuning and best practices on Big Data projects
Python [REST API], Pyspark
SPARK and MapReduce
oozie, HIVE, Impala, ELK
Yarn, Shell Scripting
Able to structure, organise and optimise large amounts of data
MS SQL Server 2008 r2/2012/2014 product stack
SPs, functions and data manipulation with SSIS