Description: *W2 Candidates Only - Cannot Support C2C*
Our client is searching for a Geospatial Data Engineer to support the design of big data solutions leveraging open source and cloud based solutions in collaboration with multiple teams across the organization (i.e. cloud analytics, data architects, business groups). The Data Engineer will participate in the building of large-scale data processing systems and API's and should be able to work with the latest open-source technologies.
A Data Engineer should embrace the challenge of dealing with terabytes or even petabytes of data daily in a high-throughput API/microservice ecosystem. A Data Engineer understands how to apply technologies to solve big data problems and to develop innovative big data solutions. The Data Engineer generally works on implementing complex projects with a focus on collecting, parsing, managing, analyzing, and making available large sets of data to turn information into insights using multiple platforms. The Data Engineer should be able to develop prototypes and proof of concepts for the selected solutions. Responsibilities
• Design, build and support of cloud and open source systems to process geospatial data assets via an API-based platform
• Partner with other internal development communities to bring needed data sets into the asset and making data available to the Bayer Enterprise and internal development communities
• Building highly scalable API's and associative architecture to support thousands of requests per second
• Being able to work across multiple teams internal/external to gather requirements and ensure project development is aligned to those requirements.
• Being able to improve the performance of the existing services and be able to identify the scope for any enhancements.
• Being able to work with parsing, managing, analyzing, and making available large sets of data to turn information into insights using multiple platforms.
• Working at all stages of the software life cycle: Proof of Concept, MVP, Production, and Deprecation Required Qualifications
• BSc degree in Computer Science or relevant job experience
• Minimum of 3-year experience with Python development language
• Knowledge in additional programming or scripting languages including but not limited to Java, Scala, and Node.js
• Experience developing HTTP APIs (Open API, REST, and/or GraphQL) which serve up data in an open-source technology, preferably in a cloud environment
• Ability to build and maintain modern cloud architecture, e.g. AWS, Google Cloud, etc.
• Experience working with PostgreSQL/PostGIS
• Experience with code versioning and dependency management systems such as GitHub, SVT, and Maven. Git experience is preferred
• Proven success utilizing Docker to build and deploy within a CI/CD Environment, preferably using Kubernetes and ArgoCD Preferred Qualifications
• MSc in Computer Science or related field
• Highly proficient (5 years) in Python
• Experience with developing HTTP APIs using common Python frameworks (Flask, Django, FastAPI)
• Proven experience (2 years) with distributed systems, e.g. Argo, Kubernetes, Spark, distributed databases, grid computing
• Proficient (4+ years) working in a Command Line Interface system e.g Docker, Argo, K8s, AWS CLI, GCloud, pSQL, SSH
• Experience with stream processing, e.g. Kafka
• Knowledge of Geoserver or other OGC standard technologies
• Experience working with customers/other developers to deliver full-stack development solutions e.g collect software, data, and timeline requirements in an Agile environment
• Demonstrated knowledge of agriculture and/or agriculture-oriented businesses
• Experience implementing complex data projects with a focus on collecting, parsing, managing, and delivery of large sets of data to turn information into insights using multiple platforms.
• Demonstrated experience adapting to new technologies
• Capable to decide on the needed hardware and software design needs and act according to the decisions. The big data engineer should be able to develop prototypes and proof of concepts for the selected solutions.
• Experience with object-oriented design, coding and testing patterns as well as experience in engineering (commercial or open source) software platforms and large-scale data infrastructures should be present
• Experience creating cloud computing solutions and web applications leveraging public and private API'sContact:
This job and many more are available through The Judge Group. Find us on the web at www.judge.com