principal

Job Description Principal Data Engineer ? Databricks | Spark | Delta Lake | PySpark | Data Lakehouse | AWS/Azure Job Description Location: Toronto Work Model: Onsite (4 days/week) Key Requirements ? 12?18 years of overall Data Engineering experience. ? 8+ years of experience with Enterprise Data Warehouse and Data Lake platforms. ? 5+ years of hands-on experience with Databricks and Apache Spark at scale. ? Strong experience modernizing legacy Cloudera platforms (CDH/CDP, Hive, HBase, Impala, Spark) to Databricks Lakehouse. ? Experience redesigning ingestion, transformation, and consumption patterns from HDFS-based architecture to cloud object storage and Delta Lake. ? Experience refactoring legacy Hive/Impala logic into PySpark and Spark SQL ELT pipelines. ? Experience ensuring data reconciliation, audit integrity, and consistency during migration. ? Experience designing and governing Enterprise Data Warehouse and Data Lake/Lakehouse architectures. ? Experience implementing layered architectures including: ? Raw/Landing Layer ? Curated/Conformed Layer ? Semantic/Consumption Layer ? Experience modernizing traditional Enterprise Data Warehouse platforms into scalable Lakehouse architectures. ? Strong experience with finance and risk data models, including: ? General Ledger ? Sub-ledger ? Financial Hierarchies ? Credit Risk Models ? Liquidity Risk Models ? Market Risk Models ? Experience enabling reporting use cases including aggregation, drill-down, and drill-back capabilities. ? Experience building and managing semantic/consumption layers for BI, reporting, and analytics. ? Ability to define business metrics, dimensions, hierarchies, and KPIs. ? Experience with Databricks SQL, Delta Tables, and dbt or similar frameworks. ? Strong experience developing and optimizing large-scale data pipelines using: ? PySpark ? Spark SQL ? Delta Lake ? Experience implementing Medallion Architecture: ? Bronze Layer ? Silver Layer ? Gold Layer ? Experience optimizing workloads using Z-ORDER, OPTIMIZE, caching, and cluster configurations. ? Experience implementing data governance, data quality frameworks, reconciliation controls, and exception handling. ? Experience establishing data lineage and metadata management. ? Knowledge of data security, access control, and compliance standards. ? Experience with cloud platforms such as AWS or Azure. ? Experience with CI/CD pipelines using: ? Git ? Terraform ? Jenkins ? Azure DevOps ? Familiarity with orchestration tools such as: ? Apache Airflow ? Databricks Workflows ? Experience with dbt is a plus. ? Ability to act as a technical authority and lead architecture decisions. ? Experience mentoring senior engineers and establishing engineering standards. ? Strong stakeholder management skills with finance, risk, analytics, and governance teams. ? Ability to translate complex data structures into business-ready insights. Nice to Have ? Experience in Banking, Financial Services, Insurance (BFSI), Capital Markets, or regulatory