Databricks (Remote)

<p>Roles &amp; responsibilities </p> <ul><li> Developing Modern Data Warehouse solutions using Databricks and AWS/ Azure Stack </li><li> Ability to provide solutions that are forward-thinking in data engineering and analytics space </li><li> Collaborating with DW/BI leads to understanding new ETL pipeline development requirements. </li><li> Triage issues to find gaps in existing pipelines and fix the issues </li><li> Work with businesses to understand the need in the reporting layer and develop a data model to fulfill </li><li>reporting needs </li><li> Help joiner team members to resolve issues and technical challenges. </li><li> Drive technical discussion with client architects and team members </li><li> Orchestrate the data pipelines in the scheduler via Airflow</li><li>Qualification &amp; experience </li><li> Bachelor's and/or master's degree in computer science or equivalent experience. </li><li> Must have a total of 6+ yrs. of IT experience and 3+ years' experience in Data warehouse/ETL projects. </li><li> Deep understanding of Star and Snowflake dimensional modeling. </li><li> Strong knowledge of Data Management principles </li><li> Good understanding of Databricks Data &amp; AI platform and Databricks Delta Lake Architecture </li><li> Should have hands-on experience in SQL, Python, and Spark (PySpark) </li><li> Candidate must have experience in AWS/ Azure stack </li><li> Desirable to have ETL with batch and streaming (Kinesis). </li><li> Experience in building ETL / data warehouse transformation processes </li><li> Experience with Apache Kafka for use with streaming data / event-based data </li><li> Experience with other Open-Source big data products Hadoop (incl. Hive, Pig, Impala) </li><li> Experience with Open Source non-relational / NoSQL data repositories (incl. MongoDB, </li><li>Cassandra, Neo4J) </li><li> Experience working with structured and unstructured data including imaging &amp; geospatial data.</li><li> Experience working in a Dev/Ops environment with tools such as Terraform, CircleCI, and GIT. </li><li> Proficiency in RDBMS, complex SQL, PL/SQL, Unix Shell Scripting, performance tuning, an troubleshooting.</li><li> Databricks Certified Data Engineer Associate/Professional Certification (Desirable). </li><li> Comfortable working in a dynamic, fast-paced, innovative environment with several ongoing </li><li>concurrent projects </li><li> Should have experience working in Agile methodology.</li><li> Strong verbal and written communication skills. </li><li> Strong analytical and problem-solving skills with a high attention to detail. Mandatory Skills: </li><li>Python/ PySpark / Spark with Azure/ AWS Databricks</li></ul> <p><br/></p>