Senior Data Engineer - Higher Education

Senior Data Engineer - Higher Education

We are currently recruiting for a Senior Data Engineer with strong Azure Services experience to join one of our Higher Education Clients on a 6 month contract.

Please note this role is Inside IR35 with hybrid working with travel to London 1 -2 days a week.

Key requirements:

• In-depth knowledge of relevant Azure services such as Azure Data Lake Storage, Azure Data Factory, Azure Synapse Analytics, Azure Databricks, Azure SQL Database, Azure Stream Analytics, etc.
• Strong Python or Scala with Spark, PySpark experience
• snowflake experience
• Experience with relational databases and NoSQL databases
• Significant experience and in-depth knowledge of creating data pipelines and associated design principles, standards, Data modelling concepts, ETL procedures, and all steps of data production process
• Experience with unit and integration testing, and data quality frameworks such as Deequ, Great Expectations or Delta Live
• Experience working with continuous integration
• Good understanding of data modelling (medallion architectures, Kimball architectures), data schemas (Avro Schema, JSON Schema) and data serialisation (Parquet, ORC, Avro)
• Advanced knowledge of Azure Data Factory for orchestrating and automating data workflows.
• Expertise in SQL and T-SQL for data querying and manipulation.
• Ability to design and implement data warehousing solutions using Azure Synapse Analytics.
• Proficiency in using Azure Databricks for data processing and analytics.
• Deep understanding of Apache Spark for large data processing.
• Expertise in setting up and optimizing Azure Data Lake Storage for storing and managing large volumes of data.
• Understanding how to deploy, manage, and monitor Azure resources using ARM templates.
• Proficiency in using the Azure Portal and Command-Line Interface for managing Azure resources.
• Knowledge of Azure security features and compliance standards.
• Familiarity with Azure Monitor and Azure Log Analytics for monitoring and logging.
• Knowledge of software engineering best practices for the full software development life cycle, including Agile methodologies, coding standards, code reviews, source control management, build processes, testing, etc

If this role is of interest to you or you would like to know more, please apply now!

Carbon60, Lorien & SRG - The Impellam Group STEM Portfolio are acting as an Employment Business in relation to this vacancy.