Data Engineering

• Responsibilities:

- Extract text data from various sources (PDF, Excel, Word, etc.).
- Process text data into specific formats according to requirements.
- Store processed data in designated databases.
- Build and automate data pipelines for automatic updates.
- Debugging and maintain data pipelines.

• Required Qualifications:

- Minimum Associate Degree (D3) in Computer Science, Information Systems, Data Science, or a related technical field.
- At least 1 years of experience as a Data Engineer
- Strong understanding of file formats (PDF, Excel, Word, etc.) and data extraction methods
- Experience in text data processing and transformation
- Proficiency in programming languages (Python, SQL)
- Knowledge of DBMS (MySQL, PostgreSQL, NoSQL)
- Strong analytical skills and attention to detail

• Preferred Qualifications:

- Knowledge of vector databases (Qdrant, Weaviate, MongoDB, etc.) and experience in using them
- Experience in building and managing data pipelines (Apache Airflow, Luigi, etc.)
- Experience with big data and related tools (Hadoop, Spark, etc.)
- Experience with cloud services (AWS, GCP, Azure)