Big Data Architect, Data Engineering Architect, contract
Architect, design, estimate, developing and deploy cutting edge software products and services that leverage large scale data ingestion, processing, storage and querying, in-stream & batch analytics for Cloud Big Data environments.
Solid experience in:
- Data related technologies, to include knowledge of Big Data Architecture Patterns and Cloud services (GCP, AWS or Azure)
- Big Data: Distributed computing, Batch and Real time processing. Spark, Hadoop, hive, Streaming (Kafka)
- Java, Scala, Python programming languages
- NoSQL database (MongoDB, Cassandra, HBase, DynamoDB, Big Table etc.)
- Detailed understanding of data pipeline automation, CI/CD, Kubernetes, infrastructure automation and SRE.
- Preferred experience on BigQuery, Spanner, GKE, Composer, Argo workflow, Helm Charts, Spinnaker, DataFlow, DataProc
- Big Data: Distributed computing, Batch and Real time processing. Spark, Hadoop/hive, Streaming (Kafka)
- Data processing framework eg Spark (Core, Streaming, PySpark SQL)
- Scalable data models addressing a wide variety of consumption patterns including random-access, sequential access including necessary optimisations like bucketing, aggregating, sharding
- Performance tuning, optimization and scaling solutions from a storage/processing standpoint
- Experience building DevOps pipelines for data solutions, including automated testing
- Data Engineering best practices, Data Management/Control, Orchestration, parallelism, Data Consistency, Data Quality, replication and building resilient data pipelines and event driven Architecture.
6 months initial, likely long term extensions
#dataarchitect #kubernetes #contractjobs #googlecloudplatform #bigquery #googledatastudio #gcp #aws #hadoop #spark #hbase #dataengineering #gcp #dataengineeringjobs #datapipelines #staffworx #recruitmentpartner #collibra #pyspark
INSIDE IR35 – this assignment will fall within scope of IR35 legislation and appropriate umbrella company should be utilised during this assignment.
This advert was posted by Staffworx Limited – a UK based recruitment consultancy supporting the global E-commerce, software & consulting sectors. Services advertised by Staffworx are those of an Agency and/or an Employment Business.
Staffworx operate a referral scheme of £500 or new iPad for each successfully referred candidate, if you know of someone suitable please forward for consideration.
