Software Development Engineer
Role: Software Dev Engineer
Duration: 6 Month Contract
Location: London
Rate: Up to £420 a day
Job summary
This project consists of transforming our AWS Redshift cluster (UDM) from a production-level system used across multiple orgs to a reporting-level system used within CF only. It consists of several tasks that focus on i) reducing the surface area, ii) deprecating and migrating core pipelines off of UDM and iii) upgrading the cluster configuration based on the new needs. In addition, third-party vendor is also expect to address company-wide mandates related to the Digital Markets Act (DMA) and security initiatives
Deliverables
UDM Migration
1. Offboard external teams
2. Offboard internal teams’ production use cases
3. Migrate key pipelines off of UDM
4. Migrate derived datasets off of UDM
5. Clean up the cluster
6. Update cluster configuration
7. Automate cluster management and maintenance
DMA and Security
1. Move UDM to private subnet
2. DMA Tagging of Customer Behavior Datasets
Skills
Candidates are expected to have 2+ years of experience in 5 out of the 6 following domains.
• Knowledge of distributed systems as it pertains to data storage and computing
• Experience with data modeling, warehousing and building ETL pipelines
• Experience with big data technologies such as: Hadoop, Hive, Spark, EMR
• Experience programming with at least one programming language such as C++, C#, Java, Python, Golang, PowerShell, Ruby
• Experience with AWS technologies like Redshift, S3, AWS Glue, EMR, Kinesis, FireHose, Lambda, and IAM roles and permissions
• Experience with non-relational databases / data stores (object storage, document or key-value stores, graph databases, column-family databases)
Reference: 53341273
Please note Reed.co.uk does not communicate with candidates via Whatsapp, and we will never ask you to provide your bank, passport or driving licence details during the application process. To stay safe in your job search and flexible work, we recommend visiting JobsAware, a non-profit, joint industry and law enforcement organisation working to combat labour market abuse. Visit the JobsAware website for information and free expert advice for safer work.
Report this jobNot quite what you are looking for? Try these similar searches
Replace a job alert
Replace a job alert
Get Job Alerts straight to your inbox
"Office Assistant jobs in London"
Your Job Alert has been created and your search saved.
'Saved search name'