Job description
Lightcast is looking for a talented Data Engineer to work on the Web Scraping team. Their primary responsibility is to design, implement, and maintain data pipelines that transform data, adding value to it in the process. In carrying out the objective of this position, a successful candidate will be committed to our values by working with gratitude, operating as a team, and partnering with our customers.
Core Responsibilities
- Create and improve processes, tools, workflows, and resilient data architecture to scrape web content.
- Manage data accuracy and quality.
- Identify and rectify any issues with breaks as well as scale scrapers as needed.
Organizational Relationships
- This position reports to the Senior Director of Data Innovation.
Knowledge, Skills, and Abilities
- Web scraping experience (Scrapy or similar technologies)
- Solid understanding of web technologies (HTML, JavaScript, CSS, XPath, JSON, etc)
- Experience with ETL and creating data pipelines
- Basic Linux and Git experience
- Python
- Scrapy Python library
- AWS/AWS Batch
- Familiarity with data processing tools (pandas, regex, SQL)
Traits
- Attention to detail
- Self-starter
- Curious
- Customer Oriented
- Organized
- Problem solving
Credentials and Experience
- 2+ years Python experience, preferred
- Professional experience with web scraping preferred
- Bachelor’s degree, preferred
Lightcast is proud to be an equal opportunity workplace and is committed to equal employment opportunity regardless of race, color, ancestry, religion, sex, national origin, sexual orientation, age, citizenship, marital status, disability, gender identity or Veteran status. Lightcast has always been, and always will be, committed to diversity, equity, and inclusion. We seek dynamic professionals from all backgrounds to join our team, and we encourage our employees to bring their authentic, original, and best selves to work.
#LI-Remote
johnandkristie.com is the go-to platform for job seekers looking for the best job postings from around the web. With a focus on quality, the platform guarantees that all job postings are from reliable sources and are up-to-date. It also offers a variety of tools to help users find the perfect job for them, such as searching by location and filtering by industry. Furthermore, johnandkristie.com provides helpful resources like resume tips and career advice to give job seekers an edge in their search. With its commitment to quality and user-friendliness, johnandkristie.com is the ideal place to find your next job.