Big Data Architect (Data Engineering)

Big Data Architect, Data Engineering Architect, contract

Architect, design, estimate, developing and deploy cutting edge software products and services that leverage large scale data ingestion, processing, storage and querying, in-stream & batch analytics for Cloud Big Data environments.

 Solid experience in:

  • Data related technologies, to include knowledge of Big Data Architecture Patterns and Cloud services (GCP, AWS or Azure)
  • Big Data: Distributed computing, Batch and Real time processing. Spark, Hadoop, hive, Streaming (Kafka)
  • Java, Scala, Python programming languages
  • NoSQL database (MongoDB, Cassandra, HBase, DynamoDB, Big Table etc.)
  • Detailed understanding of data pipeline automation, CI/CD, Kubernetes, infrastructure automation and SRE.
  • Preferred experience on BigQuery, Spanner, GKE, Composer, Argo workflow, Helm Charts, Spinnaker, DataFlow, DataProc
  • Big Data: Distributed computing, Batch and Real time processing. Spark, Hadoop/hive, Streaming (Kafka)
  • Data processing framework eg Spark (Core, Streaming, PySpark SQL)
  • Scalable data models addressing a wide variety of consumption patterns including random-access, sequential access including necessary optimisations like bucketing, aggregating, sharding
  • Performance tuning, optimization and scaling solutions from a storage/processing standpoint
  • Experience building DevOps pipelines for data solutions, including automated testing
  • Data Engineering best practices, Data Management/Control, Orchestration, parallelism, Data Consistency, Data Quality, replication and building resilient data pipelines and event driven Architecture.

6 months initial, likely long term extensions        

#dataarchitect #kubernetes #contractjobs #googlecloudplatform #bigquery #googledatastudio #gcp #aws #hadoop #spark #hbase #dataengineering #gcp #dataengineeringjobs #datapipelines #staffworx #recruitmentpartner #collibra #pyspark

INSIDE IR35 – this assignment will fall within scope of IR35 legislation and appropriate umbrella company should be utilised during this assignment.

This advert was posted by Staffworx Limited – a UK based recruitment consultancy supporting the global E-commerce, software & consulting sectors. Services advertised by Staffworx are those of an Agency and/or an Employment Business.

Staffworx operate a referral scheme of £500 or new iPad for each successfully referred candidate, if you know of someone suitable please forward for consideration.


Upload your CV/resume or any other relevant file. Max. file size: 2 MB.

Start typing and press Enter to search

FULL STACK .NET COREcypress automation