Contract: IT Data Engineer Job at in London
Senior Data Engineer Data is a strong enabler for value capture. In order to leverage this and as a key player in the Energy Trading space, the clients Gas Trading desk has developed a Data & Analytics team, on the Data Platform that they hosts on Microsoft Azure.
The Gas Desk Analytics team is looking its Data Engineering capabilities by onboarding a senior Data Engineer. This role will be a contract based opportunity and will be for an on going duration
Overview of role
The Data Engineers will work alongside the Tech Lead to build and unit test Data pipelines, data scrapes, data models in data lakes, and data warehouse.
- Work in development sprint teams under the direction of the Tech Lead
- Be responsible for development and unit testing of user stories that have been assigned to them
- Be responsible for developing robust data pipelines and data models
- Use Azure technologies for development e.g. Azure Data Factory, Azure Databricks, Azure Data Lake, Azure Synapse, Azure Analysis Service, SQL server.
- Support the creation and maintenance of the analytics infrastructure Inform the Tech Lead / Scrum Master of any risks, issues or blockers
- Participate in relevant sprint meetings e.g. daily stand-ups, retrospectives and sprint planning meetings * Maintain the required status on the chosen technology e.g. Jira * Provide SME input on technology and systems.
Roles and Responsibilities
- Manages one or more data engineering projects of moderate complexity.
- Leads or acts as key team member in defining and builds the data pipelines that will enable faster, better, data-informed decision-making within the business.
- Jointly designs and implements a new big data solution which supports high amounts and velocity of data and supports future growth.
- Identifies latest development, testing and deployment techniques to quickly deploy new releases to e.g. deploy new data pipelines and add data sources.
- Takes direct reports, when applicable, through Agile Framework and its respective tools. Influences stakeholders in adopting analyses / outcomes.
- Acts proactively to support the business to further professionalize their MI / Analytic reporting platform, using the ESSA approach. Opinions valued by business interface.
- Plays a prominent role in project related meetings etc. Utilises business partnering in order to gather, analyse and model data and key performance indicators of the highest complexity.
- Working as Data Engineering Lead to develop and deploy innovative big data platforms for advanced analytics and data processing.
- Works independently under broad managerial supervision.
Required competencies / Essential experience
- Microsoft Azure cloud services – Azure Databricks, Python, Spark
- Event Hubs, Azure Stream Analytics, Azure Data Factory, Azure Blob Storage
- Azure Data lake Store(ADLS Gen1 / Gen2), Azure Functions, Azure DevOps, Azure Sql Server
- Good communication skills Functional Experience: Energy Trading Domain; European Gas preferably.
Company Location: London