HomeAll CoursesAboutPartner
All Courses
•
Computational aspects of AI for environmental sciences
•
Challenges of large-scale data analysis and data system architectures
Back to courseBack to all courses

Lessons

Intro Part I - Data science and big data analytics

2:35

Web accessible data & data publications

8:58

Pythons request library

7:27

Some hints for good data management

10:41

The netCDF file format

6:39

The role of metadata

4:00

Work with netCDF data in Python

6:32

Intro Part II - Data science and big data analytics

1:07

Types of data in Earth system science

6:29

5 "V" of Earth system data types

10:56

How to cope with > 1 TByte of data

1:33

Intro Part III - Data science and big data analytics

1:50

Challenges of large-scale data analysis and data system architectures

6:15

Data structures, data models & data patterns

4:45

Classic design patterns

14:10

Modern design patterns

15:20

Hadoop & MapReduce

6:29

Challenges of large-scale data analysis and data system architectures

This part of the lecture describes different types of data storage systems and discusses some implications for the management of data.It covers simple file systems, databases and data warehouses, hierarchical storage architectures on HPC systems, and complex client-server architectures.

Lecturer

s

•
PD Dr. Martin Schultz
•
•

Additional Material

Additional Material

Additional Material

Git Repository with Jupyter Notes

KI:STE

About KI:STEWork PackagesResearchAI Platform

Partners

Forschungs­zentrum JülichAachen University RWTHUniversity of CologneUniversity of Bonn

Members

GitLabB2DROP

Legal

Privacy PolicyImprint