Big Data Architect
Please note - This is a 100% remote opportunity and you can work from any location.
You will be a part of Cactus Labs which is the R&D Cell of Cactus Communications. Cactus Labs is a high impact cell that works to solve complex technical and business problems that help keep us strategically competitive in the industry. We are a multi-cultural team spread across multiple countries. We work in the domain of AI/ML especially with Text (NLP - Natural Language Processing), Language Understanding, Explainable AI, Big Data, AR/VR etc.
Within Cactus Labs you will work with the Big Data team. This is a small team that manages 20-30TB of data today which is on its way to grow to 150-200Tb within the next 10-12 months. We have also designed a custom.
Your role within the Big Data team is crucial and exciting. In this role, you will lead project end to end. You will collaborate and interact with business to understand needs and also get a chance to manage and set expectations. You will be a part of planning and roadmap discussions with a team of Product Specialists, Subject Matter Experts and industry leaders. You will also get to design a Big Data pipeline that operates at scale and yet has room to grow 10X.
You may also have opportunities to advise some of the largest publishers and key players in the industry helping them realize the importance of big data and drive technical partnerships ahead
- Drive and own the product from the technical point of view as well as be the technical SME while working with Product Managers, Big Data Engineers, Big Data/Cloud Architect and Domain SMEs.
- Design and build solutions for massive scale. This requires extensive benchmarking to pick the right approach.
- Understand the data in and out, and make sense of it. You will sometimes need to draw conclusions and present it to the business users.
- Actively mentor and coach the team and help them realise the best solution to a problem
- Be independent, self-driven and self-motivated. While you will have the best people to learn from and access to various courses or training materials, we expect you to be proactive.
- Keep yourself up to date with progress in the domain. And take charge of your learning and growth. Since we work with cutting-edge technologies and are highly experimental in our approach.
Requirements (Desired skills & experience)
- 6-10 Years of experience in Big Data. Experience in the Big Data ecosystem is a must. Deep understanding of distributed computing and Big Data Ecosystem - Hadoop, MapReduce, HDFS, Spark etc
- Experience of architecting big data pipeline in Cloud Environment (AWS, Azure or GCP). You can setup, maintain and optimise big data pipelines. You have a knack for developing architectures that can scale to handle 10x data, is highly available and remain cost effective.
- You know how to setup and maintain a data lake - know about how to split data in lakes, ponds and puddle as well as when to transition data across them. You can query data from Datalake for near real-time analysis.
- Extensive hands-on knowledge of using visualization tools and creating data dashboard for easy consumption.
- Good familiarity with search engines like ElasticSearch and Bigdata warehouses systems like AWS Redshift, Google BigQuery etc.
- You understand that data is only useful when in the hands of consumers and only as successful as the consumers.
- You are eager to work with our Product Specialist to understand as well as identify various use cases and deliver on them.
- You are keen & proactive on collaborating with a team of Big Data Engineers, Cloud Architects etc who are also working towards a common vision.
- You like to work towards an end goal without a lot of handholding or supervision. You like to be hands-on with your work and do not shy away from getting on the keyboard whenever required.
- Above all, you get excited by data. You like to dive deep, mine patterns and draw conclusions. You believe in making data driven decisions.
- You like extending mentorship and coaching to the team and can work both as a team player and a leader when required.