Software Engineer

Software Engineer

21 Oct
|
ELife
|
Junagadh

21 Oct

ELife

Junagadh

Our Company:

A fast-growing start-up headquartered in San Francisco, CA, USA in the heart of Silicon Valley. We recruit worldwide as our customer base is global.

Vision: Reliable ground transportation services globally with all types of vehicles.​

Mission: Empower high quality local fleets​

Coorporate Culture: team first

- Partner-centric​
- Team collaboration,​
- Never "not my job", end-to-end ownership,​
- Continuous learning and improvement,​
- Hard-working and pragmatic,​
- Don’t be a middleman,​
- Result-driven​

Job title: Crawler Engineer

A Crawler Engineer is primarily responsible for designing and developing web crawler systems to scrape, clean, and analyze data from various platforms.





This position requires a deep understanding of how web crawlers work, familiarity with common anti-crawling techniques and countermeasures, and the ability to handle large-scale data processing.

**Primary Responsibilities**

- Design and develop efficient web crawler systems to meet business data scraping requirements.
- Conduct scraping strategy analysis on target websites and formulate optimal scraping plans.
- Maintain and optimize existing crawler systems to improve data scraping speed and accuracy.
- Clean and process scraped data to ensure data quality and availability.
- Keep track of and research the latest crawler technologies and anti-crawling mechanisms to continuously enhance the performance of crawler systems.

**Requirements**

- 5 years experience in similar positions
- Bachelor's degree or above in Computer Science or a related field, with a solid foundation in computer science.
- Proficient in Python programming, familiar with commonly used crawler frameworks (such as Scrapy, PySpider, etc.)





and information extraction techniques (such as regular expressions).
- Familiar with HTTP/HTTPS protocols, Cookie mechanisms, and web scraping principles.
- Proficient in JavaScript, XPath.
- Proficient in databases such as MySQL and BigQuery.
- Have knowledge of common anti-crawling techniques and countermeasures, able to tackle various anti-crawling challenges.
- Experience in designing and developing distributed systems, familiar with multithreading, asynchronous programming, and other technologies.
- Have good problem-solving skills and a strong teamwork spirit, able to work under pressure.
- Preference will be given to candidates with experience in scraping data from large platforms and handling massive datasets.

▶️ Software Engineer
🖊️ ELife
📍 Junagadh

Subscribe to this job alert:
Enter Your E-mail address to receive the latest job offers for: software engineer

Software Engineer

Software Engineer

Software Engineer JOB DESCRIPTION Analyze, design, and develop tests and test-automation suites. Design and develop a processing platform using various configuration management technologies. Test software development methodology in an agile e [...]
Junagadh
22 Oct
    Junagadh
    22 Oct

Senior Software Engineer

Senior Software Engineer

Senior Software Engineer Required Qualifications: 4 years of Software Engineering experience or equivalent demonstrated through one or a combination of the following: work experience, training, military experience, education Essential Qualifica [...]
Junagadh
25 Oct
    Junagadh
    25 Oct

Software Engineer

Software Engineer

Position Summary The Software Engineer (FS) will be responsible for designing, developing, optimizing and delivering both front-end and back-end software solutions, as well as managing database architecture and cloud infrastructure. This role requi [...]
Junagadh
24 Oct
    Junagadh
    24 Oct

Senior Software Engineer (Data Visualization Framework)

Senior Software Engineer (Data Visualization Framework)

Senior Software Engineer - (Data Visualization Framework) Job Overview: We, at Entiovi, are seeking a talented and experienced Senior Software Engineer with expertise in Javascript Frontend technologies, React.js/Vue.js, Three.js/Chart.js/D3.js, [...]
Junagadh
18 Oct
    Junagadh
    18 Oct
Subscribe to this job alert:
Enter Your E-mail address to receive the latest job offers for: software engineer