DAT351 Cloud and Distributed Resources for High Volume Data Processing

Contents and structure

The course presents technologies and principles for distributed computing in computer clusters, in grid systems and in the cloud. The course concentrates on challenges related to safe and efficient utilisation of computing resources managed by heterogeneous operators, including protection concerns between the project owner and facility management. Software technology used for such systems is applied and configured. Virtualization is covered as a method to obtain task distribution on a global scale.

Learning Outcome

Upon completion of the course the candidate should be able to:

• discuss challenges and solutions for high volume data processing.
• explain the philosophy of cloud and grid computing.
• identify tasks well suited for the different distributed computing models.
• assess selected research papers in the field of high volume data processing.
• explain the different cloud service models.
• describe the different hypervisor models used for virtualization.
• explain the MapReduce programming model.

• define and monitor job management, storage management and security in a grid system.
• design and implement applications of Service Oriented Computing at a global scale.
• design, implement and run applications on a MapReduce framework.
• design, implement and run tasks through a computer clustering management platform.

• evaluate and apply distributed computing resources using textual and graphical interfaces.
• revise application software to make it suitable for distributed computing.

Entry requirements

General admission requirements for the study programme.

Recommended previous knowledge

Experience with using a Unix/Linux operating system.

Teaching methods

Lectures, practical work in lab, presentation of papers and project work.

Compulsory learning activities

2-4 assignments in the form of written reports.

The assignments must be submitted within set deadlines and must be approved before examination can take place.

Approved assignments are valid for the examination semester and 2 following semesters.

Assessment

The course has an examination in two parts: an Grading according to the A-F scale based on an oral exam and a the project report. The project report counts for 30% of will have a weight of 30 % in the final grade and the oral exam counts for 70% of the final grade.

Both parts must get a passing grade in order to get a final grade for the course. In case one of the parts gets a failing grade, that part can be taken as a re-sitting/postponed exam.

Grading scale is A-F where F is fail.

Course reductions

• PCS951 - Skyløsninger og Distribuerte Dataressurser for Høg-Volum Dataprosessering - Reduction: 10 studypoints