Databricks Platform Administrator

At SunnyData, a leading Databricks technology partner, our mission is to empower customers with scalable architectures, robust data engineering pipelines, seamless data consumption layers, and advanced ML and AI applications. As a Databricks Platform Administrator you will manage and optimize the Databricks environment, ensuring a reliable and well-governed platform for analytics, ML, and production workloads, while working closely with the Data Platform team to scale and integrate the ecosystem.

The Impact You Will Have:

• Databricks Administration & Operations: Own, optimize, and secure the Databricks environment, ensuring scalable, reliable, and well-monitored performance for data and ML workloads.

• Data Platform & Pipeline Support: Enable and optimize data and ML workflows in Databricks, enhancing self-service capabilities and supporting Unity Catalog for governance and discovery.

• Security, Compliance & Governance: Implement and maintain Databricks security, access controls, and compliance practices, ensuring least-privilege governance and alignment with enterprise standards.

• Cross-Functional Collaboration: Partner with data and ML teams to improve the Databricks platform, drive best practices and automation, and coordinate with DevOps on cloud infrastructure and integrations.

Required Experience:

• 5+ years of experience administering or engineering on Databricks, or in a closely related data platform role.

• Strong working knowledge of Databricks workspace administration - cluster policies, permissions, Unity Catalog, Jobs/Workflows, and networking.

• Familiarity with AWS services that underpin Databricks deployments (S3, IAM, VPC, EC2).

• Experience with data pipeline and orchestration tools such as dbt, Airflow, Fivetran, or Delta Live Tables.

• Proficiency in Python and SQL. Familiarity with Spark is expected.

• Understanding of modern data governance and access control patterns - Unity Catalog, RBAC, data classification.

Preferred

• Experience in financial services, investment management, or other highly regulated industries.

• Experience with Snowflake and AWS engineering.

• Hands-on experience with MLflow, model serving, or feature store capabilities within Databricks.

• Familiarity with IaC tools (Terraform) for managing Databricks resources programmatically.

• Hands-on experience with AI Engineering and LLMOps tools - LLM observability, eval pipelines, building and supporting agentic workflows.

• Experience with cost optimization strategies for Databricks (cluster policies, spot instances, photon, serverless SQL).

• Knowledge of distributed compute frameworks (Spark, Ray, Dask).

Education

• Bachelor's degree or higher in a STEM field, required.

Our Commitment to Diversity and Inclusion

At SunnyData, we are committed to fostering a diverse and inclusive culture where everyone can excel. We take great care to ensure that our hiring practices are inclusive and meet equal employment opportunity standards. Individuals looking for employment at SunnyData are considered without regard to age, color, disability, ethnicity, family or marital status, gender identity or expression, language, national origin, physical and mental ability, political affiliation, race, religion, sexual orientation, socio-economic status, veteran status, and other protected characteristics.

Why Join SunnyData?

• Innovative Environment: Work with cutting-edge technologies and industry leaders in data engineering and AI.

• Customer Impact: Make a real difference in how businesses leverage data for strategic decision-making.

• Career Growth: Opportunities for professional development and career advancement.

• Collaborative Culture: Join a supportive team that values collaboration and knowledge sharing.

If you are passionate about data engineering and enjoy engaging with clients to solve their most challenging problems, we would love to hear from you. Apply today and become a key player in SunnyData's success story.