[Remote] Data Engineer – Google Cloud Platform
Note: The job is a remote job and is open to candidates in USA. Dice is seeking a highly skilled Data Engineer with deep expertise in Google Cloud Platform and modern data architecture. The ideal candidate will design and develop scalable data pipelines, implement Medallion Architecture, and develop enterprise-grade solutions using BigQuery, PySpark, Dataflow, Airflow, and Java.
Responsibilities
- Design and develop scalable batch and real-time data pipelines on Google Cloud Platform
- Implement Medallion Architecture (Bronze, Silver, Gold layers)
- Build high-performance data transformations using Python, PySpark, and Java
- Develop and optimize complex SQL queries for analytics workloads
- Work extensively with BigQuery for large-scale data processing and optimization
- Develop and deploy pipelines using Cloud Dataflow
- Orchestrate workflows using Cloud Composer (Apache Airflow)
- Manage storage and lifecycle using Google Cloud Storage (GCS)
- Implement CI/CD pipelines and version control practices
- Ensure governance, security, and access control using Google Cloud Platform IAM
- Optimize data solutions for performance, scalability, and reliability
Skills
- Strong hands-on experience with Google Cloud Platform (Google Cloud Platform)
- Expertise in BigQuery (Partitioning, Clustering, Query Optimization)
- Proven implementation experience with Medallion Architecture
- Strong Python and PySpark programming skills
- Hands-on Java Development experience (Mandatory)
- Advanced SQL skills (Joins, Window Functions, Performance Tuning)
- Experience with Cloud Dataflow
- Experience with Cloud Composer (Airflow)
- Experience with Google Cloud Storage (GCS)
- CI/CD Pipelines and Git
- Google Cloud Platform IAM and cloud security best practices
- Data Lake and Data Warehousing concepts
- Real-time streaming frameworks
- Data governance and data quality experience
- Agile/Scrum environment experience
Company Overview