Data Engineer

Job Role: Data Engineer

Location: TX, CA, NJ
Job Type: Full-Time / Contract

Responsibilities:

• Develop and maintain scalable ETL/ELT pipelines for efficient data processing.
• Build and manage data lakes and data warehouses on cloud and on-premise platforms.
• Work with structured and unstructured data, ensuring accuracy and reliability.
• Optimize data ingestion, storage, and retrieval for performance improvements.
• Ensure data governance, security, and compliance across all processes.
• Collaborate with Data Scientists, Analysts, and Engineers to integrate data into applications.
• Automate workflows and support real-time data streaming architectures.
• Troubleshoot and resolve data issues to maintain system efficiency.

Required Skills & Experience:

• 9+ years of experience in Data Engineering, ETL development, and Big Data processing.
• Proficiency in SQL, Python, Scala, or Java for data manipulation and automation.
• Experience with cloud-based data platforms (AWS, Azure, GCP) such as AWS Glue, Redshift, Snowflake, BigQuery, and Databricks.
• Knowledge of Big Data frameworks like Hadoop, Spark, Kafka, or Flink.
• Expertise in Data Warehousing (Snowflake, Redshift, BigQuery, Synapse Analytics).
• Strong understanding of database modeling, indexing, and query optimization.
• Familiarity with CI/CD, Infrastructure as Code (Terraform, CloudFormation), and containerization tools (Docker, Kubernetes).
• Hands-on experience with data orchestration tools like Apache Airflow, Prefect, or Luigi.
• Excellent problem-solving and analytical skills.

Preferred Skills (Nice to Have):

• Experience with real-time streaming (Kafka, Kinesis, Spark Streaming, Flink).
• Familiarity with Graph Databases (Neo4j, ArangoDB, Amazon Neptune).
• Exposure to Machine Learning Pipelines & MLOps.
• Understanding of DataOps best practices.

How to Apply:

If you or someone you know is interested in this opportunity, please send resumes to garima.patankar@softstandard.com.

References are highly appreciated!