SC Cleared Data Engineer Inside IR35 6 Months Hybrid

Data Engineer/Architect - Inside IR35 - Hybrid - London - SC Clearance Needed

Location: London
Work Arrangement: Hybrid
Security Clearance: SC Cleared Needed
Sector: Public Sector Consultancy

About the Role: The Cross-Product Performance Analytics Team at the Home Office's Migration, Borders, and Technology Portfolio is seeking an experienced and innovative Data Engineer/Architect. The successful candidate will lead the design, development, and implementation of a data lake to support our data integration and analytics initiatives. This role involves working closely with data scientists, analysts, and other stakeholders to ensure the efficient processing, storage, and retrieval of large datasets. The individual will also be instrumental in developing scalable data pipelines and optimizing our data architecture for advanced analytics and reporting.

Key Responsibilities:

Data Lake Design and Implementation:

• Lead the architecture, design, and deployment of a data lake to centralize and manage large volumes of data from various sources.
• Develop scalable and efficient data pipelines to facilitate data ingestion, transformation, and storage.
• Ensure the data lake architecture supports advanced analytics, machine learning, and reporting requirements.

Advanced Analytics Support:

• Collaborate with data scientists and analysts to support the development and deployment of advanced analytics models and algorithms.
• Ensure the data lake infrastructure supports real-time and batch processing requirements for analytics and reporting.
• Contribute to the implementation and optimization of dashboards and reporting tools, including those based on ServiceNow and DORA 4 metrics.

Data Integration and Management:

• Design and implement ETL (Extract, Transform, Load) processes to integrate data from multiple sources into the data lake.
• Ensure data quality, consistency, and reliability through robust data validation and cleansing procedures.
• Optimize data storage and retrieval processes to enhance performance and scalability.

Collaboration and Communication:

• Work closely with data scientists, data analysts, and other stakeholders to understand data requirements and deliver solutions that meet their needs.
• Communicate complex technical concepts and solutions to non-technical stakeholders.

Qualifications and Skills:

• Proven experience in data engineering and architecture, particularly in designing and implementing data lakes.
• Strong understanding of data pipelines, ETL processes, and data integration.
• Experience with advanced analytics, machine learning, and reporting tools.
• Ability to work collaboratively with technical and non-technical stakeholders.
• Excellent communication skills, with the ability to convey complex technical information clearly.

Data Engineer/Architect - Inside IR35 - Hybrid - London - SC Clearance Needed