CSC-343 Course Notes

• Course Introduction (with Docker examples)
  • Docker Summary
• Linux Tutorial
  • Corrections / Notes
• Introduction to Hadoop (Available on Blackboard)
• Hadoop Architecture (Available on Blackboard)
• HDFS
• Python Basics
• MapReduce
  • Word Count Example
  • MapReduce Worksheet
• Relational Databases
• Sqoop (Available on Blackboard)
• Intro to Impala and Hive (Available on Blackboard)
• Modeling Data with Impala and Hive (Available on Blackboard)
• Introduction to Spark (Available on Blackboard)
  • Functional Programming in Python
• RDDs in Spark (Available on Blackboard)
  • Dictionaries and JSON in Python
• Working with RDDs in Spark (Available on Blackboard)
• Aggregating Data with Pair RDDs (Available on Blackboard)
  • Word Count Example, PySpark Style
• Spark Parallel Processing (Available on Blackboard)
  • Partitions
• Spark RDD Persistence (Available on Blackboard)
  • Persistence Example
• Spark Algorithms (Available on Blackboard)
  • Page Rank
  • K-Means
• SparkSQL Example
• Wrap Up