Big Data Engineer
- 5+ years experience in Data Engineering and Warehousing
- Expertise in streaming frameworks and data ingestion technologies
- Experience in building Spark SQL and managing structured and unstructured data
About the job Big Data Engineer
Job Description:
· 5+ years experience working in Data Engineering and Warehousing.
· 3 -5 years’ experience integrating data into analytical platforms
· Experience in ingestion technologies (e.g. Swoop, flume), processing technologies (Spark/Scala), and storage (e.g. HDFS, HBase, Hive)
· Experience in data profiling, source-target mappings, ETL development, SQL optimization, testing, and implementation
· Expertise in streaming frameworks (Kafka/Spark Streaming/Storm) essential
· Experience in building Spark SQL and Spark Data Frame API-based applications.
· Experience managing structured and unstructured data types
· Experience in requirements engineering, solution architecture, design, and development/deployment
· Experience in creating big data or analytics IT solution
· Track record of implementing databases and data access middleware and high-volume batch and (near) real-time processing.
We vet all employer accounts and do our best to keep job ads safe, but scams can still occur. Be cautious when sharing personal information — never provide financial details or make payments during the application process. For extra security, use the Apply button on our site when proceeding.
Skills
0 of 20 matchedPyramid Global Technologies