Data Engineer

Company Details

The world’s first human intelligence platform.

Description

Streetbees is building the world’s first platform to understand human behaviour and decision making. We make human data accessible, anywhere in the world, by connecting brands with real people on the ground in over 100 markets to gather real-time in-the-moment insights at scale. We distil this data using Natural Language Understanding to uncover not just what they do, but also why they do it, and what drives them – and predict what they may do next.

Having recently raised our Series B investment led by Lakestar, Streetbees’ partners with world leading brands across the FMCG, Media and Entertainment, Retail & Insurance sectors and our clients include Unilever, PepsiCo, Sony Pictures, Carlsberg, Santander, Ikea, Vodafone and L’Oreal.

Streetbees has a very strong engineering team and platform, combining the best in machine learning, data management pipelines, web platforms and mobile apps. We work with Python for our data science and machine learning, Ruby and Elixir with ReactJS for our web apps and APIs and middleware, and Swift and Kotlin (our Android codebase is 100% Kotlin). Everything is entirely hosted in AWS, all repos in Github, everything containerised and continuously deployed.

An opportunity exists for a Data Engineer to join our team, particularly those who find excitement in a 10x scaling of the platform. The collection and analysis of data at scale is key to Streetbees technology activities as a leading ‘applied AI’ company – devops practices and scalable infrastructure are fundamental to how we achieve this. We are looking for enthusiasts to contribute to building something truly great in this space.

The Data Engineer will have strong technical knowledge relative to their years of experience but also a keen appreciation for the role devops plays in engineering practices and culture. We are looking for engineers at both mid and senior experience levels as we are more driven by a colleagues contribution and skills than length of service, and as a fast growing start-up we have flexibility in exact skills requirements in order to build the best team. Successful candidates will probably have significant experience in some of these areas; data engineering (e.g. Python, Kafka, Spark, Aurora Postgres, Airflow etc) and container engineering (e.g. ECS, Docker, Kubernetes etc).

This is a fully hands on position. Experience with Python, Spark and SQL will be essential while experience with AWS, CircleCI (or other automated build platforms), Terraform, Ansible and Datadog are nice to have. This is not a role for a pure sysadmin or platform engineer, it is a software engineering role with platform infrastructure elements and would particularly suit engineers who have experience with programming in Python but who are also comfortable with creating infrastructure resources using Terraform. Engineers who have a self-motivated interest in developing their own devops practices and supporting the wider engineering community at Streetbees will find a collaborative environment and a supportive blameless culture.

Responsibilities

  • Developing and maintaining data pipelines for our Data Lake, ML training loops and data operations workflows
  • Working closely with the Data Operations and BI teams to support their activities with data transformations and new data sources
  • Playing an integral role in implementing our Data Strategy
  • Hands-on design, development and support of infrastructure in collaboration with the rest of the platform engineering team. Active pull request participation, branch management, code reviews etc
  • Creating and managing infrastructure as code using Terraform and Atlantis
  • Investigating bugs and performance issues and providing support to the product engineering teams
  • Working closely with developers and product managers to ensure that data infrastructure requirements are implemented correctly and efficiently
  • Being an active part of the group of engineers reviewing and making key tech design decisions, and coordinating work and operations on our platform architecture
  • Sharing knowledge within the engineering team and data analysts using presentations, demos and pair programming
  • Working on production deployment of Machine Learning AI solutions (DataOps for machine learning is a new area for the industry, with little current conventional best practice, Streetbees expertise in ML creates a real opportunity to do leading-edge DataOps work productionising that)
  • Keeping up to date with the latest evolutions in engineering best practice and technologies

Requirements

  • Experience with Python including pyspark, pandas, numpy is essential
  • Knowledge of Ruby would be beneficial but not essential
  • Experience with Spark, AWS Glue, Athena, Delta Lake, Redshift
  • Experience managing relational databases (we use Aurora Postgres)
  • Experienced with the AWS cloud platform
  • Experience with containers and scheduling tools (we use Docker on ECS)
  • Experience managing infrastructure as code with Terraform
  • Experience working with CI/CD pipeline tools such as CircleCI, Jenkins etc

Benefits

At Streetbees, your company benefits will include:

  1. A 24-day paid holiday allowance with the option to rollover up to three days a year if desired
  2. Flexible working hours – we trust people to manage their time and to focus on wider results; so if you need to work from home to finish a project or leave at 5 on a Tuesday for Choir practice, ensure your manager knows in advance, but feel free to take it.
  3. An external training or conference budget of £500 per year to use for topics agreed with your line manager.
  4. A pension scheme – our new pension scheme means that Streetbees will start paying a percentage of your salary towards your pension.
  5. The Streetbees Culture Champions – any employee will have the opportunity to plan and manage a variety of Streetbees funded socials and activities for the team. Socials and day outs can involve anything from rock climbing or bowling to company meals or pub crawls
  6. Wednesday Lunch – We all come together for lunch every Wednesday with a different cuisine each week, paid by the company. Vegetarian and vegan options always provided!
  7. Fully Stocked Kitchen – Breakfast is on us every morning and if you need a coffee or a snack at any point throughout the day, please be our guest.
  8. Friday Catch-ups – Beers, wine, soft drinks and snacks provided as we celebrate the week’s successes and welcome newcomers.
  9. Paid volunteering days – one day per quarter with a registered charity of your choice
  10. Games Room – Nintendo Switch with big screen TV + various games
  11. Audible subscriptions – the company is more than happy to pay for audio-books subscriptions of your choice on Amazon as we encourage wider and continued learning among our team
  12. Free beer and Prosecco in the office!

Tagged as: data operations, tech design

Visit us on LinkedInVisit us on FacebookVisit us on Twitter