AWS Tech Lead with Databricks Job at TMS, Raritan, NJ

NWg2dlJqQW5BQmptN3JpWXdzcTN3b295K3c9PQ==
  • TMS
  • Raritan, NJ

Job Description

Role: AWS Tech Lead with Databricks

Duration: Long Term

Location: Raritan, NJ- fully onsite

Overview:

An AWS Tech Lead with Databricks SME (Subject Matter Expert) knowledge is responsible for designing, deploying, and managing cloud-based infrastructure on AWS and supporting data solutions using Databricks. The role involves leading a team to ensure scalable, secure, and cost-efficient cloud services while integrating data pipelines and analytics solutions.

Roles and Responsibilities:

Cloud Infrastructure Management:

- Design, implement, and maintain AWS cloud solutions, including VPCs, EC2 instances, S3 storage, RDS, and networking configurations.

- Ensure high availability, disaster recovery, and backup solutions for cloud environments.

- Implement Infrastructure as Code (IaC) using tools like Terraform, AWS CloudFormation, or CDK.

Data Solutions with Databricks:

- Serve as a Databricks SME, guiding the architecture and implementation of ETL pipelines, data lakes, and machine learning workflows using Databricks on AWS.

- Optimize Databricks clusters for performance, cost, and security.

- Integrate Databricks with other AWS services like S3, Redshift, and Glue for seamless data processing.

Leadership and Team Collaboration:

- Lead a team of cloud engineers, providing mentorship and technical guidance.

- Collaborate with cross-functional teams including data scientists, analysts, and developers to understand requirements and translate them into scalable cloud and data architectures.

- Manage project timelines, ensuring timely delivery of cloud and data solutions.

Security and Compliance:

- Implement best practices for cloud security, including identity and access management (IAM), encryption, and VPC security.

- Ensure compliance with industry standards (e.g., GDPR, HIPAA) and internal security policies.

- Conduct regular security audits and optimize cloud configurations to reduce risk.

Performance Optimization and Monitoring:

- Set up monitoring, logging, and alerting using AWS CloudWatch, Datadog, or similar tools.

- Analyze system performance and provide recommendations for improvement.

- Troubleshoot complex issues related to cloud infrastructure, data pipelines, and Databricks clusters.

Technical Skills:

- Strong expertise in AWS services (EC2, S3, Lambda, RDS, VPC, CloudFormation, IAM, etc.).

- Proficiency with Databricks, including building and managing data pipelines, data lakes, and machine learning models.

- Experience with Infrastructure as Code (IaC) tools like Terraform or AWS CloudFormation.

- Knowledge of big data technologies like Apache Spark, Hadoop, or Delta Lake.

- Proficient in programming and scripting languages (e.g., Python, SQL, Scala).

- Experience with CI/CD pipelines for deploying cloud infrastructure.

Soft Skills:

- Strong leadership and project management skills.

- Excellent problem-solving and analytical skills.

- Ability to work collaboratively with diverse teams.

- Strong communication skills, with the ability to explain technical concepts to non-technical stakeholders.

Certifications:

Required:

- AWS Certified Solutions Architect (Associate or Professional)

- Databricks Certified Associate Developer for Apache Spark

Preferred:

- AWS Certified DevOps Engineer Professional

- AWS Certified Big Data Specialty or AWS Certified Data Analytics Specialty

- Databricks Certified Professional Data Engineer

Job Tags

Similar Jobs

Schlossmann Subaru City Of Milwaukee

Automotive Detailer Job at Schlossmann Subaru City Of Milwaukee

SCHLOSSMANN SUBARU CITY IS SEEKING EXPERIENCED AUTO DETAILERS FOR OUR DETAIL/RECONDITIONING DEPARTMENT. Full-time position are available with flat rate compensation. What you do:* Clean, wax, polish, refurbish and detail vehicles* Inspect finished products to...

Kaneka North America

Junior Buyer Job at Kaneka North America

 ...vendor invoices, and other payment documents in the SAP system. Monitors requisition entry inconsistencies, enforces company purchasing policies and informs management of deviations when appropriate. Utilize the Fairmarkit platform to Identify and develop reductions... 

Acme Transportation Company

Local CDL A Company Driver - 1st Shift Job at Acme Transportation Company

 ...Acme Transportation is looking for company drivers for 1st shift home daily positions. This is a 90% intermodal driving position with...  ...Intermodal experience is a must!~100% No Touch. ~ CDL A HOURLY first shift home daily driving position. ~ The rate is $22.50/per... 

Kadiko

Creative Director, Copywriter - Healthcare Brand Agency Job at Kadiko

 ...passionate about creating meaningful connections between our clients and their audiences. We are looking for a talented and visionary Creative Director/Copywriter to join our dynamic team. Why Apply at Kadiko# Impact: we transform businesses, brands and lives.#... 

SUN HOSPITALITY RESORT SERVICES

Director of Housekeeping Job at SUN HOSPITALITY RESORT SERVICES

 ...Sun Hospitality is a full-range housekeeping provider offering services to the vacation resort industry. We are committed to creating a...  ...grow their hospitality career with us. We are looking for a Director of Housekeeping to join our team at our location in Charleston...