Applied Data Engineering

Intermediate 5 days Course & Certification


Upon completion of this course, participants will be familiarized with all major aspects of Big Data Analytics and its ecosystems. Participants will be able to develop, construct, test and maintain architectures such as databases and large-scale processing systems, perform batch and real-time streaming analytics on structured and unstructured data, execute professional data management, as well as create visualizations and dashboards. This course will provide an in-depth, stepwise hands-on experience.

Target Audience

This course is suitable for candidates who are interested in knowing more about the Big Data Analytics ecosystem, looking to become full-fledged Data Engineers, and acquiring some technical know-how in the area of Data Science. Participants should preferably have some knowledge in Python Programming.


Introduction to Big Data
Features of Data Engineering
What is ETL/ELT and Best Practices
Metadata Management
Consolidating Multiple Data Sources
Data Ingestion, Cleansing & Transformation
Hadoop Architecture and Ecosystem
Flat Files Ingestion into Hadoop
RDBMS & Hadoop Integration
Hive Data Processing
Interactive Query using Impala
Log Files Handling and Processing
Data Web Scraping
Introduction to Spark
Processing Data using PySpark
Spark Data Query
Real-time Data Analytics in Spark Streaming
Troubleshooting ETL Jobs
Performance Optimization


Optional certification available


24 - 26 Oct, 1 - 2 Nov 2023
5 Days


At this juncture, all classes for this course will be carried out virtually. The face-to-face option will be made available in the future.
We reserve the right to cancel or reschedule any training session. In the event of a cancellation or if you are unable to attend a rescheduled training session, we may choose to refund all of your paid fees or credit such amounts towards the next available training session.
You may request to have your training rescheduled to a later date, provided that you have notified us at least 10 working days prior to the training. Course fees will not be refunded if you notify us in less than 10 working days, or do not show up on the day of the training.
If are you unable to attend, you may designate a representative from your organization to attend prior to the commencement of the training at no additional cost.
You may cancel your training and get refunded, subject to conditions. Course fees paid shall be refunded strictly on the basis that you have notified us at least 10 working days prior to the training. No refunds will be made if you give notice in less than 10 working days.
Yes, please contact us here ( if you would like to request for a private class or in-house training arrangement for this course.
Yes, this course is HRD Corp (Human Resource Development Corp of Malaysia) claimable. Please contact us here ( if you would like to utilize your HRD Corp levy (instead of paying in cash) for this course.
Yes, this course includes multiple hands-on sessions for you to learn and practice what you are learning so you can have the best learning experience possible.
After attending this course, you can opt to sit for an exam to get yourself certified in a separate arrangement and cost. We strongly encourage you to get certified since having industry recorganized certification will definitely add value to your career. Please contact us here ( if you would like to opt for the optional professional certification.
Applied Data Engineering