Data Engineer-Duke Health Technology Systems-Full Time-DHTS-Durham

Duke University
Closing date
Oct 22, 2021

View more

Administrative Jobs
Technology, Analysts & Programming

Job Details

Duke Health Technology Solutions (DHTS) is a robust, specialized division of Duke University Health System dedicated to the development and management of enterprise IT systems. A 2018 ‘Most Wired’ health system, Duke is nationally recognized for IT and information management as the first healthcare system to achieve the Davies Award – highest honor by the Healthcare Information and Management Systems Society (HIMSS) – for inpatient, ambulatory, and analytics health information technology capabilities. Duke was also the first to achieve Stage 7 HIMSS for Hospital EMR, Ambulatory EMR, and Analytics adoption. Our employees are among the top-skilled IT experts in the Triangle. We partner with leading scholars, clinicians, and researchers across Duke University and Duke Health to develop innovative technologies that support our mission of delivering tomorrow's healthcare today.

Primary Role: This position reports to the DHTS-Analytics Center of Excellence, Senior Manager, Data Engineering. This individual will be primarily responsible for the development of data integration and delivery pipelines while also expanding the content stored within Duke’s Enterprise Analytics Platform. These solutions should capitalize on technologies to improve the value of analytical data, improve effectiveness of information stewardship, and streamline the flow of data in the organization.Solutions should focus on using state of the art data and analytics tools including traditional and near real-time data warehousing, big-data, relational and document based databases using both extract, load, transform (ELT) toolsets as well as REST APIs and FHIR. The ideal candidate will be comfortable with data science platforms with proven experience leveraging DevOps and Automation/Orchestration tools.

Essential Tasks/Responsibilities

    Create and maintain optimal data pipeline architecture.Assemble large, complex data sets that meet functional / non-functional business requirements.Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.Recommend design of analytics solutions which improves data integration, data quality, and data delivery with an eye towards re-useable componentsArticulate differences, advantages, and disadvantages between architectural solution methodsWork with Agile team members to document and execute test plans for data loading and data validation scripts. Support the code promotion process through development and production as required by using standard CI/CD processes.Develop, implement and maintain schedule/dependency logic for automated ETL processing.Develop monitoring, logging, and error notification processes to ensure data is updated as expected and processing metrics reported. Participate in the creation and maintenance of standards for coding, documentation, error handling, error notification, logging, etc. Accountable for conforming to established architectural, developmental, and operational standards and practices including creation of metadata.Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.Evaluate and recommend database development toolsAssist in application and database performance tuningParticipate in system architecture designCreate data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.Work with data and analytics experts to strive for greater functionality in our data systems.Share trouble-shooting and maintenance duties

Education: Bachelor's degree in a related field, or four years of equivalent technical experience required

Required Experience: We are looking for a candidate with 5+ years of experience in a Data Engineer role who should also have experience using the following software/tools:

    Experience with Cloud-based analytic platforms such as Azure Synapse.Experience with relational SQL and NoSQL databases, including Postgres and CosmosDB in Azure.Experience with data pipeline and workflow management tools such as Airflow.Experience with the FHIR model for healthcare a plusExperience with stream-processing systems such as Spark-Streaming.Experience with object-oriented/object function scripting languages such as Python or Java.

Required Skills:

    Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.Strong analytic skills related to working with unstructured datasets.Build processes supporting data transformation, data structures, metadata, dependency and workload management.A successful history of manipulating, processing and extracting value from large disconnected datasets.Working knowledge of message queuing, stream processing, and highly scalable ‘big data’ data stores.Strong project management and organizational skills.Experience supporting and working with cross-functional teams in a dynamic environment.

Desired Skills:

    Knowledge of APIs, API Integration, and API Management Working knowledge of DevOps & Automation/Orchestration Experience with the FHIR specificationKnowledge of open source software solutions and open source as a business model Experience in data science using Python or R Technical breadth across application development, enterprise architecture, or application integrationUnderstanding of Agile methodologyPrior experience in health care related field is a plus

Duke is an Affirmative Action/Equal Opportunity Employer committed to providing employment opportunity without regard to an individual's age, color, disability, gender, gender expression, gender identity, genetic information, national origin, race, religion, sex, sexual orientation, or veteran status.

Duke aspires to create a community built on collaboration, innovation, creativity, and belonging. Our collective success depends on the robust exchange of ideas—an exchange that is best when the rich diversity of our perspectives, backgrounds, and experiences flourishes. To achieve this exchange, it is essential that all members of the community feel secure and welcome, that the contributions of all individuals are respected, and that all voices are heard. All members of our community have a responsibility to uphold these values.

Essential Physical Job Functions: Certain jobs at Duke University and Duke University Health System may include essentialjob functions that require specific physical and/or mental abilities. Additional information and provision for requests for reasonable accommodation will be provided by each hiring department.


Read our Diversity Profile History

Duke University was created in 1924 by James Buchanan Duke as a memorial to his father, Washington Duke. The Dukes, a Durham family that built a worldwide financial empire in the manufacture of tobacco products and developed electricity production in the Carolinas, long had been interested in Trinity College. Trinity traced its roots to 1838 in nearby Randolph County when local Methodist and Quaker communities opened Union Institute. The school, then named Trinity College, moved to Durham in 1892, where Benjamin Newton Duke served as a primary benefactor and link with the Duke family until his death in 1929. In December 1924, the provisions of indenture by Benjamin’s brother, James B. Duke, created the family philanthropic foundation, The Duke Endowment, which provided for the expansion of Trinity College into Duke University.Duke Campus

As a result of the Duke gift, Trinity underwent both physical and academic expansion. The original Durham campus became known as East Campus when it was rebuilt in stately Georgian architecture. West Campus, Gothic in style and dominated by the soaring 210-foot tower of Duke Chapel, opened in 1930. East Campus served as home of the Woman's College of Duke University until 1972, when the men's and women's undergraduate colleges merged. Both men and women undergraduates now enroll in either the Trinity College of Arts & Sciences or the Pratt School of Engineering. In 1995, East Campus became the home for all first-year students.

Duke maintains a historic affiliation with the United Methodist Church.

Home of the Blue Devils, Duke University has about 13,000 undergraduate and graduate students and a world-class faculty helping to expand the frontiers of knowledge. The university has a strong commitment to applying knowledge in service to society, both near its North Carolina campus and around the world.

Mission Statement

Duke Science"James B. Duke's founding Indenture of Duke University directed the members of the University to 'provide real leadership in the educational world' by choosing individuals of 'outstanding character, ability, and vision' to serve as its officers, trustees and faculty; by carefully selecting students of 'character, determination and application;' and by pursuing those areas of teaching and scholarship that would 'most help to develop our resources, increase our wisdom, and promote human happiness.'

“To these ends, the mission of Duke University is to provide a superior liberal education to undergraduate students, attending not only to their intellectual growth but also to their development as adults committed to high ethical standards and full participation as leaders in their communities; to prepare future members of the learned professions for lives of skilled and ethical service by providing excellent graduate and professional education; to advance the frontiers of knowledge and contribute boldly to the international community of scholarship; to promote an intellectual environment built on a commitment to free and open inquiry; to help those who suffer, cure disease, and promote health, through sophisticated medical research and thoughtful patient care; to provide wide ranging educational opportunities, on and beyond our campuses, for traditional students, active professionals and life-long learners using the power of information technologies; and to promote a deep appreciation for the range of human difference and potential, a sense of the obligations and rewards of citizenship, and a commitment to learning, freedom and truth.Duke Meeting

 “By pursuing these objectives with vision and integrity, Duke University seeks to engage the mind, elevate the spirit, and stimulate the best effort of all who are associated with the University; to contribute in diverse ways to the local community, the state, the nation and the world; and to attain and maintain a place of real leadership in all that we do.”

Find Us

Get job alerts

Create a job alert and receive personalised job recommendations straight to your inbox.

Create alert