I was spending 2-3 hours a day on LinkedIn. I was scrolling and reading the job description, figuring out if it fits my skill set. I did research for a typical salary for the role, wrote cover letters, etc. I did this 30 times. Most of them didn't even match! I decided that I needed to build a pipeline to do this for me.
The Pipeline
I broke it up into stages. Each one filters and enriches the data, so by the next stage, it's less and less noisy.
LinkedIn Scrape → Filter → AI Match Score → Salary Estimate → Cover Letter → Apply
500+ ~80 ~25 ~25 ~25 ~25
500 jobs get scraped from the site, and by the end, only 25 of them make it to the application stage. That's 95% noise reduction.
Scraping LinkedIn Without Getting Banned
LinkedIn hates scraping. They detect scrapers through mouse movement patterns, scroll speed, and other things in your session. My solution:
# Selenium with undetected-chromedriver
from undetected_chromedriver import Chrome
driver = Chrome(headless=False) # headless gets detected faster
driver.get("https://linkedin.com/jobs")
# Human-like delays between actions
import random
time.sleep(random.uniform(2.1, 4.7))
The key decisions that helped me stay undetected:
- Never ran headless (LinkedIn has fingerprinting)
- Randomly delaying between each action between 2-5 seconds
- Rotating sessions every 50 views of jobs
- Exponential backoff when rate limited
The Selector Problem
Almost every time LinkedIn does a deploy, they change the names of the classes in the DOM. One day, it'll be a .jobs-search-results__list-item. The next day, it'll be a .scaffold-layout__list-detail.
I built a fallback system.
SELECTORS = {
"job_card": [
"[data-job-id]", # Most stable — data attribute
"[aria-label*='job']", # ARIA labels change less often
".job-card-container", # Class name — least stable
]
}
def find_element(driver, key):
for selector in SELECTORS[key]:
try:
return driver.find_element(By.CSS_SELECTOR, selector)
except NoSuchElementException:
continue
raise ElementNotFound(f"All selectors failed for {key}")
Data attributes and aria attributes are a bit more stable than class names. They're used to describe the role that a particular section of HTML plays. So it's unlikely that the developers at LinkedIn will just start changing them out of nowhere.
AI Match Scoring
I matched each job to your resume using GPT-4o-mini (not GPT-4 because that's too slow and too expensive, especially if you have to do it 80+ times a run). This is the prompt I used:
score_prompt = f"""
Score this job against the candidate's resume.
Return a JSON object with:
- match_score (0-100)
- matching_skills (list)
- missing_skills (list)
- deal_breakers (list)
- recommendation (apply/skip/maybe)
Resume: {resume_text}
Job: {job_description}
"""
I used GPT-4o-mini because it reliably returns JSON in ~0.3 seconds per job. It cost about $0.02 to score 80 jobs.
4-Layer Salary Estimation
I am most proud of this. Salary data from any single source is not reliable. So I cross reference it across 4 different sources:
- Salary data from public profiles on Coresignal
- Data scraped from LinkedIn Salary Insights (if available, scraped from the job listing)
- GPT-4 estimates: role, company, location, seniority
- Algo: regression (similar job roles in the same geography)
Final Estimate: We take a weighted average based on how reliable our sources are, and if 3 of the 4 estimates agree within 15% we're confident.
Cover Letter Generator
Template cover letters do not work. Not asking GPT to "write a cover letter" either, because it sounds like every other AI-generated cover letter.
The solution: Give LLM your resume, job description, and constraint (example: reference X project that is similar to this role).
The Result
500+ jobs scanned daily. 25 high match applications with cover letters. This used to take 3 hours. Now it takes 10 minutes to review.