Company is seeking a highly skilled and experienced Data engineer to join our team.
The ideal candidate will have a proven track record of designing, developing and deploying data-driven solutions using Python and big data tools such as Apache Spark.
Responsibilities:
Develop data-driven applications and solutions using Python and Spark.
Utilize advanced Python concepts such as multiprocessing and asynchronous functions to optimize web scraping speed and efficiency.
Design, develop and maintain scalable and efficient ETL pipelines.
Write clean, efficient, and reusable code that adheres to coding standards and best practices.
Collaborate with cross-functional teams including product owners, data scientists, and software engineers to identify opportunities for product improvements.
Troubleshoot and optimize existing systems to ensure maximum performance and scalability.
Requirements:
At least 5 years of experience as a Python developer with a focus on data analysis.
Expertise in advanced Python concepts such as multiprocessing and asynchronous functions.Strong experience in big data technologies such as Apache Spark.
Experience with data warehousing, ETL, and data modeling.
Experience with distributed computing and cloud platforms such as GCP or AWS
Excellent problem-solving skills and attention to detail.