Description

We are seeking a Databricks Data Engineer with strong hands-on experience in Python and PySpark to support a fast-paced, high-impact delivery environment. This role requires someone who can quickly step in, take ownership, and deliver scalable data solutions with minimal ramp-up.

Key Responsibilities

· Design, develop, and optimize data pipelines and ETL workflows using Databricks

· Build scalable data processing solutions leveraging PySpark and Python

· Work with large-scale datasets across cloud platforms (AWS and/or Azure)

· Implement and maintain Medallion Architecture (Bronze, Silver, Gold layers)

· Ensure data quality, performance optimization, and reliability of data pipelines

· Collaborate with cross-functional teams to support data-driven initiatives

Required Qualifications

· 3 to 5 years of experience in Databricks is required.

· Strong hands-on experience with Databricks, Python, PySpark, and SQL

· Proven experience building and optimizing ETL pipelines

· Experience working with large-scale data environments

· Solid understanding of cloud platforms (AWS and/or Azure)

· Familiarity with Medallion architecture (Delta Lake)

· Ability to work independently in fast-paced, deadline-driven environments

What We’re Looking For

· Someone who can hit the ground running with minimal onboarding

· Strong sense of ownership and accountability

· Ability to deliver under tight timelines without compromising quality

· Proactive problem-solver with a delivery-first mindset Nice to Have

· Experience with Delta Lake

· Exposure to data governance and data quality frameworks

· Prior experience in federal or regulated environments

Preferred Additional Qualifications:

· Experience with Delta Lake

· Exposure to data governance and data quality frameworks

· Prior experience in federal or regulated environments

Requirements

Required Qualifications: :

Education:

Bachelor's degree in information systems, computer science, or other related field. Will accept a suitable combination of education, training, or experience

Clearance:

Clearance: Federal Health Agency within the U.S. Department of Health and Human Services Public Trust clearance

Onsite Requirement:

Location: Hybrid - locals to DMV area

Professional Experience

3 to 5 years of experience in Databricks is required.
Strong hands-on experience with Databricks, Python, PySpark, and SQL
Proven experience building and optimizing ETL pipelines
Experience working with large-scale data environments
Solid understanding of cloud platforms (AWS and/or Azure)
Familiarity with Medallion architecture (Delta Lake)
Ability to work independently in fast-paced, deadline-driven environments

Diaconia LLC AI Jobs

Latest AI Jobs at Diaconia LLC

Databricks Data Engineer