Data Connection Engineer (Partial Remote) with Security Clearance
Overview We are currently seeking a Data Connections Engineer to work Partial/Remote in Crystal City, VA (2-3 days per week on site)This role requires active Secret ClearanceData is absolutely the lifeblood of BigBear.
ai and our data engineers are the doctors administering it! Our data engineers support data collection, ingestion, validation, and loading of optimized data in the appropriate data storesThey work on a team made up of analyst(s), developer(s), data scientist(s), and a product lead and everyone on the team collaborates in support of a specific missionWorking directly with the analyst(s) and the product lead, the data engineer identifies and implements solutions for the data requirements, including building pipelines to collect data from disparate, external sources, implementing rules to validate that expected data is received, cleansed, transformed, massaged and in an optimized output format for the data storeAs a Data Connections Engineer, you will be responsible for designing, implementing, and maintaining efficient data connections and integrations within the organization's data ecosystemThis position requires experience with Python, PySpark, SQL, API's and SFG RelaysWhat you will doo Design data integration solutions to connect various data sources and systems using Python, Spark, and Databricks.
o Define data connection requirements, data flow diagrams, and integration patterns.
o Develop and maintain scalable data stores.
o Perform data transformation and enrichment operations to ensure data quality, consistency, and compatibility across different systems.
o Implement rules to validate that expected data is received, cleansed, transformed, massaged and in an optimized output format for the data store.
o Develop custom APIs (Application Programming Interfaces) to enable secure and efficient data exchange between internal and external systems.
o Integrate APIs with data pipelines to facilitate real-time data updates and synchronization.
o Implement data security measures and protocols to protect sensitive information during data connections and integrations.
o Ensure compliance with relevant data privacy regulations and industry standards.
o Identify and resolve performance bottlenecks in data connections and integration workflows.
o Optimize data processing, network utilization, and resource allocation to enhance overall system performanceWhat you need to haveo Bachelor's Degree and 5
years of experienceo Clearance:
Must possess and maintain an active Secret clearanceo Experience with a modern programming language such as Python or Java (strong Python skills preferred)o Experience developing data pipelines using modern Big Data ETL technologieso Experience working in a big data and cloud environmento Experience with and an understanding of data pain points and challenges across respective DoD client joint commandso Experience developing solutions for high volume, low latency applicationso In-depth knowledge and hands-on experience with Apache Spark, including Spark SQL, Spark Streaming, and Spark DataFrame APIo Proficiency in working with Databricks platform for large-scale data processing, data integration, and analytics.
o Experience with RESTful API development, implementation, and integration What we'd like you to haveo Experience working in an agile development environmento Ability to quickly learn technical concepts and communicate with multiple functional groupso Ability to display a positive, can-do attitude to solve the challenges of tomorrowo Possession of excellent verbal and written communication skillso A person who appreciates the opportunity to be independent, creative and challenged.
o An individual with a curious mind, passionate about solving problems quickly and bringing innovative ideas to the tableAbout BigBear.
ai BigBear.
ai delivers AI-powered analytics and cyber engineering solutions to support mission-critical operations and decision-making in complex, real-world environmentsBigBear.
ai's customers, which include the US Intelligence Community, Department of Defense, the US Federal Government, as well as customers in manufacturing, healthcare, commercial space, and other sectors, rely on BigBear.
ai's solutions to see and shape their world through reliable, predictive insights and goal-oriented adviceHeadquartered in Columbia, Maryland, BigBear.
ai is a global, public company traded on the NYSE under the symbol BBAIFor more information, please visit:
http:
//bigbear.
ai/ and follow BigBear.
ai on Twitter:
@BigBearai.
Recommended Skills Api Agile Methodology Apache Spark Artificial Intelligence Big Data Computer Networks Estimated Salary: $20 to $28 per hour based on qualifications.
ai and our data engineers are the doctors administering it! Our data engineers support data collection, ingestion, validation, and loading of optimized data in the appropriate data storesThey work on a team made up of analyst(s), developer(s), data scientist(s), and a product lead and everyone on the team collaborates in support of a specific missionWorking directly with the analyst(s) and the product lead, the data engineer identifies and implements solutions for the data requirements, including building pipelines to collect data from disparate, external sources, implementing rules to validate that expected data is received, cleansed, transformed, massaged and in an optimized output format for the data storeAs a Data Connections Engineer, you will be responsible for designing, implementing, and maintaining efficient data connections and integrations within the organization's data ecosystemThis position requires experience with Python, PySpark, SQL, API's and SFG RelaysWhat you will doo Design data integration solutions to connect various data sources and systems using Python, Spark, and Databricks.
o Define data connection requirements, data flow diagrams, and integration patterns.
o Develop and maintain scalable data stores.
o Perform data transformation and enrichment operations to ensure data quality, consistency, and compatibility across different systems.
o Implement rules to validate that expected data is received, cleansed, transformed, massaged and in an optimized output format for the data store.
o Develop custom APIs (Application Programming Interfaces) to enable secure and efficient data exchange between internal and external systems.
o Integrate APIs with data pipelines to facilitate real-time data updates and synchronization.
o Implement data security measures and protocols to protect sensitive information during data connections and integrations.
o Ensure compliance with relevant data privacy regulations and industry standards.
o Identify and resolve performance bottlenecks in data connections and integration workflows.
o Optimize data processing, network utilization, and resource allocation to enhance overall system performanceWhat you need to haveo Bachelor's Degree and 5
years of experienceo Clearance:
Must possess and maintain an active Secret clearanceo Experience with a modern programming language such as Python or Java (strong Python skills preferred)o Experience developing data pipelines using modern Big Data ETL technologieso Experience working in a big data and cloud environmento Experience with and an understanding of data pain points and challenges across respective DoD client joint commandso Experience developing solutions for high volume, low latency applicationso In-depth knowledge and hands-on experience with Apache Spark, including Spark SQL, Spark Streaming, and Spark DataFrame APIo Proficiency in working with Databricks platform for large-scale data processing, data integration, and analytics.
o Experience with RESTful API development, implementation, and integration What we'd like you to haveo Experience working in an agile development environmento Ability to quickly learn technical concepts and communicate with multiple functional groupso Ability to display a positive, can-do attitude to solve the challenges of tomorrowo Possession of excellent verbal and written communication skillso A person who appreciates the opportunity to be independent, creative and challenged.
o An individual with a curious mind, passionate about solving problems quickly and bringing innovative ideas to the tableAbout BigBear.
ai BigBear.
ai delivers AI-powered analytics and cyber engineering solutions to support mission-critical operations and decision-making in complex, real-world environmentsBigBear.
ai's customers, which include the US Intelligence Community, Department of Defense, the US Federal Government, as well as customers in manufacturing, healthcare, commercial space, and other sectors, rely on BigBear.
ai's solutions to see and shape their world through reliable, predictive insights and goal-oriented adviceHeadquartered in Columbia, Maryland, BigBear.
ai is a global, public company traded on the NYSE under the symbol BBAIFor more information, please visit:
http:
//bigbear.
ai/ and follow BigBear.
ai on Twitter:
@BigBearai.
Recommended Skills Api Agile Methodology Apache Spark Artificial Intelligence Big Data Computer Networks Estimated Salary: $20 to $28 per hour based on qualifications.
|
|
 | |
