Lead Data Engineer – Batch and Stream Processing
Company Name:-
GM Financial
Job Location:-
Arlington, TX 76014 (East area)
Job Summary:-
Job detailsJob TypeFull-timeFull Job DescriptionOverview:
We are expanding our efforts into complementary data technologies for decision support in areas of ingesting and processing large data sets including data commonly referred to as semi-structured or unstructured data.
Our interests are in enabling data science and search based applications on large and low latent data sets in both a batch and streaming context for processing.
To that end, this role will engage with team counterparts in exploring and deploying technologies for creating data sets using a combination of batch and streaming transformation processes.
These data sets support both off-line and in-line machine learning training and model execution.
Other data sets support search engine based analytics.
Exploration and deployment of technologies activities include identifying opportunities that impact business strategy, selecting data solutions software, and defining hardware requirements based on business requirements.
Responsibility also includes coding, testing, and documentation of new or modified scalable analytic data systems including automation for deployment and monitoring.
This role participates along with team counterparts to architect an end-to-end framework developed on a group of core data technologies.
Other aspects of the role include developing standards and processes for data engineering projects and initiatives
Responsibilities:
Job Duties
Evaluate, research, experiment with batch and streaming data engineering technologies in a lab to keep pace with industry innovation while assessing business impact and viability for use cases associated with efforts in hand
Work with data engineering related groups to inform on and showcase capabilities of emerging technologies and to enable the adoption of these new technologies and associated techniques
Define and refine processes and procedures for the data engineering practice
Work closely with data scientists, data architects, ETL developers, other IT counterparts and business partners to identify, capture, collect and format data from the external sources, internal systems and the data warehouse
Code, test, deploy, monitor, document, and troubleshoot data engineering processing and as
FOR MORE DETAILS CLICK BELOW LINK [convertful id=”110657″]
