Ryan Specialty Logo

Ryan Specialty

Lead Azure Data Engineer

Posted 20 Days Ago
Be an Early Applicant
In-Office or Remote
Hiring Remotely in Chicago, IL
140K-175K Annually
Expert/Leader
In-Office or Remote
Hiring Remotely in Chicago, IL
140K-175K Annually
Expert/Leader
Leads Azure data engineering initiatives, designing scalable data pipelines, ETL processes, data architectures, and governance solutions. Manages Azure Data Factory, Databricks, Delta Parquet, Azure Data Lake Storage, Profisee MDM, Azure Purview, SSIS, SSRS, Tableau, and SQL Server optimization. Implements real-time data integration, change data capture, CI/CD, automation, and reporting solutions while ensuring data quality, performance, reliability, and compliance. Provides technical leadership and mentors junior data engineering staff.
The summary above was generated by AI

Position Summary

The Lead Azure Data Engineer will be reporting to VP of Data Engineering and will be responsible for implementing data solutions and collaborating with Business Analyst, Data Analyst, IT Teams and key stake holders. This role will lead projects and coach junior staff on build and optimize data pipelines, integrating data from various sources, ETL (Extract, Transform, Load) processes and ensuring data quality and consistency. The Lead will ensure that our data engineering practices are aligned with the latest technologies and industry best practices, focusing on performance, reliability, and efficiency.

What will your job entail?

Job Responsibilities:
• Provide technical leadership in data architecture and engineering, collaborating with solution stakeholders to align data solutions with business objectives.
• Mentor less experienced data engineers and team members, fostering a collaborative and innovative work environment.
• Explore and implement new technologies and practices in data engineering to continually improve our data processes and infrastructure.
• Manage and enhance complex SSIS packages for data integration and ETL processes, along with SSRS reports for business intelligence.
• Design, build, and maintain scalable and efficient data pipelines using Azure Data Factory, implementing both batch and real-time data integration processes.
• Develop and optimize Databricks notebooks for big data processing and analytics in Azure Databricks environments.
• Implement and manage data storage solutions using Delta Parquet files in Azure Data Lake Storage, optimizing for query performance and cost-efficiency.
• Integrate and configure Profisee Master Data Management tools within Azure, focusing on data governance and quality.
• Set up and manage Change Data Capture processes for real-time data synchronization in Azure data platforms.
• Implement data cataloging, classification, and lineage using Azure Purview, developing data governance policies, and ensuring compliance.
• Enhance and support Tableau reporting solutions, developing complex dashboards, and integrating with Azure data services.
• Drive automation and process optimization using Azure services and scripting and implement CI/CD pipelines using Azure DevOps.
• Lead SQL Server database optimization and performance tuning efforts.
• Lead, Coach and Mentor junior staff on projects.
Work Experience and Education:
• Bachelor’s degree required, Master’s degree preferred; Computer science, Information Technology or Engineering is preferred. Any other related discipline or commensurate work experience considered.
• 12+ years of experience in Data Engineering with a Bachelor’s degree, or 8 years of experience with a Master’s degree is preferred.
• Subject Matter Expert in SQL Server Management, Azure Data Factory Development, Databricks Engineering, Delta Parquet File Management, and SQL Server Integration Services.
• Proficiency in Profisee MDM Integration, Change Data Capture implementation, and data quality and governance with Azure Purview.
• Proven experience in Tableau and BI Reporting, and strong skills in Automation and Process
Licenses & Certifications:
• Advanced Data Analytics Certifications (ADAC), SAS Statistical Business Analyst Professional Certification (SBAPC), or Cloudera Certified Professional (CCP) Data Engineer a plus.
Skills:
Technical/Functional Skills:
• Big Data Analytics
• Leadership
• Data Governance
• Data Modeling
• Data Processing
• Data Warehousing
• Database Management
• Knowledge of ETL processes
• Machine Learning
• Predictive Analysis
• Programming Languages
Behavioral Skills:
• Critical Thinker
• Communication
• Relationship Building
• Effective Communication
• Time Management
• Problem Solving
• Attention to Detail

   *****Applicants must be authorized to work for any employer in the U.S. We are unable to sponsor or take over sponsorship of an employment visa at this time*****

Ryan Specialty is an Equal Opportunity Employer. We are committed to building and sustaining a diverse workforce throughout the organization. Our vision is an inclusive and equitable workplace where all employees are valued for and evaluated on their performance and contributions. Differences in race, creed, color, religious beliefs, physical or mental capabilities, gender identity or expression, sexual orientation, and many other characteristics bring together varied perspectives and add value to the service we provide our clients, trading partners, and communities. This policy extends to all aspects of our employment practices, including but not limited to, recruiting, hiring, discipline, firing, promoting, transferring, compensation, benefits, training, leaves of absence, and other terms, conditions, and benefits of employment.

How We Support Our Teammates 

Ryan Specialty seeks to offer our employees a comprehensive and best-in-class benefits package that helps them — and their family members — achieve their physical, financial, and emotional well-being goals. In addition to paid time off for company holidays, vacation, sick and personal days, Ryan offers paid parental leave, mental health services and more.

The target salary range for this position is $140,000.00 - $175,000.00 annually.

The wage range for this role considers many factors, such as training, transferable skills, work experience, licensure and certification, business needs, and market demands. The pay range is subject to change and may be modified in the future. Full-time roles are eligible for bonuses and benefits. For additional information on Ryan Specialty Total Rewards, visit our website https://benefits.ryansg.com/. 

 

We provide individuals with disabilities reasonable accommodations to participate in the job application or interview process, to perform essential job functions, and to receive other benefits and privileges of employment in accordance with applicable law. Please contact us to request an accommodation at [email protected] 

 

The above is intended to describe this job's general requirements. It is not to be construed as an exhaustive statement of duties, responsibilities, or physical requirements. Nothing in this job description restricts management's right to assign or reassign duties and responsibilities to this job at any time. Reasonable accommodations may be made to enable individuals with disabilities to perform essential functions. 

Similar Jobs

An Hour Ago
Easy Apply
In-Office or Remote
United States
Easy Apply
Mid level
Mid level
Cloud • Security • Software • Cybersecurity • Automation
Lead the SSCS Add-On engineering team to develop security capabilities, managing delivery, team growth, and quality in a collaborative environment.
Top Skills: Ci/CdDevsecopsSigstoreSlsa
3 Hours Ago
In-Office or Remote
United States
90K-105K Annually
Mid level
90K-105K Annually
Mid level
Big Data • Information Technology • Software • Analytics • Energy
Administers and analyzes sales compensation programs, including commission payout processing, data auditing, dispute resolution, plan modeling, reporting, and dashboard maintenance. Partners with sales, finance, and HR stakeholders to ensure accurate, compliant compensation operations and support plan improvements. Requires strong Excel modeling, CRM and incentive compensation platform experience, analytical ability, communication skills, and independent judgment.
Top Skills: CaptivateiqExcelPerformioSalesforceVaricentXactly
3 Hours Ago
Remote or Hybrid
United States
17-25 Hourly
Entry level
17-25 Hourly
Entry level
Artificial Intelligence • Automotive • Greentech • Information Technology • Machine Learning • Software • Cybersecurity
Processes, codes, audits, and indexes service-provider invoices for fleet billing operations. Responsibilities include resolving invoice discrepancies, verifying approval limits, completing WACC requirements, processing rebills, performing VMRS coding and costing, managing purchase-order inquiries, and responding to overflow calls. The role requires accurate high-volume data entry, attention to detail, and communication with service providers and internal teams.
Top Skills: TmcareVmrs

What you need to know about the NYC Tech Scene

As the undisputed financial capital of the world, New York City is an epicenter of startup funding activity. The city has a thriving fintech scene and is a major player in verticals ranging from AI to biotech, cybersecurity and digital media. It also has universities like NYU, Columbia and Cornell Tech attracting students and researchers from across the globe, providing the ecosystem with a constant influx of world-class talent. And its East Coast location and three international airports make it a perfect spot for European companies establishing a foothold in the United States.

Key Facts About NYC Tech

  • Number of Tech Workers: 549,200; 6% of overall workforce (2024 CompTIA survey)
  • Major Tech Employers: Capgemini, Bloomberg, IBM, Spotify
  • Key Industries: Artificial intelligence, Fintech
  • Funding Landscape: $25.5 billion in venture capital funding in 2024 (Pitchbook)
  • Notable Investors: Greycroft, Thrive Capital, Union Square Ventures, FirstMark Capital, Tiger Global Management, Tribeca Venture Partners, Insight Partners, Two Sigma Ventures
  • Research Centers and Universities: Columbia University, New York University, Fordham University, CUNY, AI Now Institute, Flatiron Institute, C.N. Yang Institute for Theoretical Physics, NASA Space Radiation Laboratory

Sign up now Access later

Create Free Account

Please log in or sign up to report this job.

Create Free Account