Microsoft Azure Data Engineer Job at Purple Drive, Boston, MA

N05vemExNXpsUnJHUUpGTVdlWVRTMnIyNHc9PQ==
  • Purple Drive
  • Boston, MA

Job Description

Microsoft Azure~Digital

6-9 yrs
Data Engineer - Databricks
- Must have at least 8+ years of extensive Data Engineering Experience in Databricks, ETL/ELT using Data pipelines, SQL/Procedure experience.
- Must have 8+ Years of hands-on development proficiency in implementing Databricks solutions using Scala and Spark with Data frames and Notebooks/SQL.
- Must have 8+ Years of hands-on development and performance tuning/enhancement experience in Scala/PySpark and SQL.
- Must have 8+ Years of hands-on development experience in working on Azure Cloud.
- Must have strong experience with Databricks, including developing and optimizing Spark jobs, data transformations, and data processing workflows.
- Must have experience and a good understanding on the below topics
-- Databricks Delta Lake storage
-- Unity Catalog
-- Autoloader
-- Lakehouse Architecture
-- Medallion Architecture
-- Databricks Serverless Options.
-- Dataset and Data frame
-- Databricks Clusters Configuration and Sizing
-- Databricks Job Orchestration
-- Spark Structured Streaming
- Must have experience with complex and large volume data streaming and data transformations using Azure Databricks.
- Must have extensive experience in Datawarehouse/Data Lakehouse implementations.
- Must have experience with monitoring, debugging, and resolving issues in live Databricks jobs.
- Must have strong hands-on expertise in troubleshooting DevOps pipelines, Azure, and AKS services

- The candidate should be able to provide Data Ingestion, Streaming and Transformations solutions and do development work.
- Experience in Apache Big Data implementations with programming proficiency like PySpark, R or Java prior to Databricks is advantageous.
- Good to have experience with containerization technologies such as Docker and Kubernetes.
- Strong critical thinking, communication, and problem-solving skills is essential.

Job Tags

Similar Jobs

International Health and Medical Services

Behavioral Health Specialist Job at International Health and Medical Services

 ...capabilities. With 12,000 staff (including 5,200 medical and behavioral health providers) our services include the design, deployment...  ...Medical Services is looking for an experienced Behavioral Health Specialists to assess, diagnose, and treat mental health disorders among... 

Munger Physical Therapy

Medical Receptionist Job at Munger Physical Therapy

 ...Join Our Team at Munger Physical Therapy Part-Time Medical Receptionist (1520 hours/week) Munger Physical Therapy is looking for a warm, organized, and dependable Medical Receptionist to become part of our compassionate and growing team. This is a part-time position... 

Luxury Bath Technologies

Brand Ambassador Job at Luxury Bath Technologies

 ...Brand Ambassador Best part-time job in Jacksonville!30 year old company with offices in three states is expanding to Jacksonville and needs 2 people to add to our marketing department. With more than 25 years in business, Quality Craftsmen is one of the fastest... 

BoatJunkie Sales & Service Inc

Marine Technician - Yamaha/ Mercury Job at BoatJunkie Sales & Service Inc

We are looking for a highly knowledgeable and experienced Yamaha or Mercury Marine Technician. At least 3-5 years of proven track record in marine industry is a must. Main responsibilities: 1. Examine and inspect boat and discuss the nature and extent of damage or... 

Delan Associates, Inc

Plumbing Designer- Senior Job at Delan Associates, Inc

 ...Work Location: Greenville, SC (In office, Onsite) Job Title: Plumbing Designer (S3D) Responsibilities: Able to work within a team and complete areas as the schedule indicates. Good communication skills are a must. Review existing models and work through...