Lead Data Engineer
Award winning company and global industry leader has a newly opened opportunity for a Lead Data Engineer . This is an opportunity to join a highly respected research organization who has consistently delivered insightful products and insights throughout the financial sector with customers spanning over 100 countries. The Lead Data Engineer will lead a team in the full cycle development of a new, innovative product offering using current cloud-based technology. This position will offer a competitive compensation package including a base salary up to $150,000, an annual bonus, and excellent mobility to grow in both responsibility and compensation. Although partial remote work may be offered, candidate’s must be flexible with working Post Covid from the company’s headquarters located in Baltimore, MD. In addition to having professional experience with the below responsibilities, ideal candidates will have strong Python skills, major cloud platform experience, and industry exposure within the financial services sector.
Lead Data Engineer - Job Description
- Help lead a team in developing a proprietary analytical information product using the company’s current cloud-based technology stack.
- Perform functions as a self-motivated, enthusiastic and passionate software engineer to develop data solutions with financial data used in product applications.
- Analyze and provide insight for areas of improvement for cloud-based data architecture
- Design and develop software using cutting edge technologies consisting of data pipelines, ETL, big data, machine learning and cloud-based development methodologies
- Play a key role in the development team to build high-quality, high-performance, scalable code to ingest data from various financial data sources.
- Produce technical design documents and conduct technical walkthroughs
- Collaborate effectively with technical and non-technical stakeholders and respond to and resolve production issues.
- Utilize excellent Python coding skills and understanding of Python best practices including experience in designing and developing Web Services and RESTFUL APIs using Python
- Develop data in PostgreSQL or comparable relational database management system (RDBMS)
- Write complex queries using SQL and NoSQL statements and write clean, maintainable code, including unit tests (TDD)
- Use methodologies including Elasticsearch and machine learning frameworks, Natural Language Processing (NLP), as well as conduct performance tuning
- Utilize various data-store technologies or data processing framework in the designing, architecting and building reliable data pipelines
- Work with large structured and unstructured data sets, various ETL techniques, and integrate and fuse data from multiple data sources
- Exhibit strong passion for innovation and delivering value to customers including being highly effective with working cross-functional in a dynamic team environment
Lead Data Engineer – Job Requirements
- 6 or more years of professional experience in software engineering, design and development of scalable data applications
- 3 or more years of cloud-based development within a major cloud provider such as AWS, Azure, or Google.
- Strong Python coding skills and understanding of Python best practices. Ability to write clean, maintainable code, including unit tests (TDD)
- Utilize cloud managed services like AWS Data Pipeline, AWS Aurora, AWS Glue, AWS Step functions, AWS Lambda, AWS DynamoDB, MongoDB
- Strong experience with API development, micro-services pattern, cloud technologies and managed services preferably AWS, Big Data and Analytics
- Experience in designing and developing Web Services and RESTFUL APIs using Python
- Development with data in PostgreSQL or comparable relational database management system (RDBMS). Ability to write complex queries using SQL and NoSQL statements
- Experience with Elasticsearch and machine learning frameworks, Natural Language Processing (NLP)
- Experience with version control tools such as Git, GitHub, GitLab or similar
- Strong understanding of data structures, designing, architecting and building reliable data pipelines, various ETL techniques, integrating / fusing data from multiple data sources
- Experience or knowledge of financial data or investment industry highly preferred