Data Engineering Technical Lead, contract, to £900 day (inside IR35)
Data Engineering Lead Architect, VP Data Engineering
Data, Cloud, Batch and Streaming based data solutions, providing strategic and tactical direction to teams and customers especially in the areas of marketing and technology.
Should have a strong programming background in technologies like Java, Scala, Python along with Spark and other related computing frameworks.
As a data engineering practitioner, you will also have a strong point of view on and understanding of build vs. buy, performance considerations, hosting, business intelligence and reporting & analytics. Ideally, you will also have experience in integrating data with marketing scenarios like segmentation, targeting, consumer 360 view etc.
Your profile
- Big Data technologies and expertise in cloud related data services (AWS/Azure/GCP)
- Led technical Architecture, Design and Delivery of Big Data and Cloud Data solutions (AWS, Azure, GCP) for multiple projects
- Setup best design patterns, coding practices, code review process, automation and quality guidelines and processes
- Expert in data ingestion, distributed data processing (batch and streaming) and programming languages preferably in Java/Scala and/or Python as secondary language, distributed messaging and ingestion frameworks (Kafka, Pulsar, Pub/Sub etc)
- End to end architecture including Analytics, ML and Activation tools in overall Data-driven Digital Business Transformation (DBT) and Marketing Transformation programs
- Experience with NoSQL databases (Cassandra/HBase/MongoDB/ElasticSearch/Neo4j) and scalable analytical data stores like Snowflake, BigQuery, Redshift, Teradata
- Knowledge of scalable data models that address a wide variety of consumption patterns including random-access, sequential access including necessary optimizations like bucketing, aggregating, sharding.
- Experience of Performance tuning, optimisation and scaling solutions from a storage/processing standpoint
- Good understanding of Continuous Integration and Continuous Delivery (CI/CD) using Cloud based DevOps services or Jenkins/Bamboo, Maven, Junit, SonarQube, Terraform (one-click infrastructure setup), Kubernetes, containerization, optimisation
- Good understanding of Data Governance, Data Security, Data Cataloguing and Data Lineage concepts (any tools experience in these areas like Collibra is preferred)
- Have led data audits/assessment, defining data strategy and provide consulting skills to the clients
- Lead proposals (RFPs) from solution, architecture, estimation and framework standpoint
- Exhibit thought leadership in Data Engineering eg writing blogs, creating PoV s, possess knowledge of industry trends, attending/presenting in internal/external technical forums, mentorship etc
- Excellent communication, presentation and collaboration skills
- Lead/participate in Data CoE initiatives eg building accelerators, knowledge sharing sessions, coaching/mentoring team members
INSIDE IR35 – this assignment will fall within scope of IR35 legislation and appropriate umbrella company should be utilised during this assignment.
#dataengineer #sparkstreaming #apachespark #pyspark #apacheairflow #contractjobs #workfromhome #remote
