Independent distributed/backend builder
Job Posting Skill Demand Analyzer Engine & REST API
Validated an asynchronous Python and PySpark engine on 500,000 real posting records: 411,135 canonical postings, 156,491 skill matches across 73 canonical skills, and 0.738ms p95 over 1,000 populated sequential warm-cache Python-skill requests.
I built the asynchronous processing service, FastAPI endpoints, Redis cache-aside layer, Spark partitioning strategy, PostgreSQL indexes, Docker packaging, automated GitHub Actions tests, and a Next.js/React results explorer.
Local 500K convenience sample; 17.77% heuristic duplicate candidates and 73 matched skills. No production SLA or optimization speedup claimed.
- 01FastAPIAsync run submission
- 02RedisQueue + cache-aside
- 03PySpark500,000 → 411,135 rows
- 04Skill extraction156,491 matches · 73 skills
- 05PostgreSQLIndexed + idempotent
- 06Metrics API0.738ms p95 locally
- 07Next.js + ReactAudited snapshot explorer