• 1 Overview of the Project
    • 1.1 Introduction
    • 1.2 Availability of Data
    • 1.3 Contents of this Document
    • 1.4 Acknowledgments
    • 1.5 Instructions for generating this bookdown document
  • 2 Engagement Overview Dashboard Overview
    • 2.1 Data Cleaning Pipeline
    • 2.2 Visualization Reasoning and Caveats
  • 3 Engagement Overview Dashboard Functions
    • 3.1 Wrangling Functions
      • 3.1.1 read_dirt_engagement_data
      • 3.1.2 clean_engagement_data
      • 3.1.3 write_tidy_engagement_data
      • 3.1.4 wrangle_overview_engagement
    • 3.2 Server Functions
      • 3.2.1 filter_chapter_overview
      • 3.2.2 get_module_vector
      • 3.2.3 create_module_name
      • 3.2.4 get_nactive
      • 3.2.5 join_engagement_item
      • 3.2.6 get_module_nactive
      • 3.2.7 make_engagement_eiffel_tower
  • 4 General Demographics Overview
    • 4.1 Data Cleaning Pipeline
    • 4.2 Visualization Reasoning and Caveats:
  • 5 General Demographics Dashboard Functions
    • 5.1 Wrangling Functions
      • 5.1.1 wrangle_general
      • 5.1.2 obtain_raw_general_data
      • 5.1.3 obtain_language_info
      • 5.1.4 obtain_country_info
      • 5.1.5 prepare_general_data
      • 5.1.6 write_general_data
    • 5.2 Server Functions
      • 5.2.1 convert_loe
      • 5.2.2 get_loe_df
      • 5.2.3 get_loe_plot
      • 5.2.4 get_age_df
      • 5.2.5 get_age_plot
      • 5.2.6 get_top_country_df
      • 5.2.7 get_country_plot
      • 5.2.8 get_top_language_df
      • 5.2.9 get_language_plot
  • 6 Link and Page Overview
    • 6.1 Data Cleaning Pipeline
    • 6.2 Visualization Reasoning and Caveats
  • 7 Link and Page Dashboard Functions
    • 7.1 Wrangling Functions
      • 7.1.1 read_course_axis
      • 7.1.2 read_link_dirt
      • 7.1.3 hash_username
      • 7.1.4 set_activity_level
      • 7.1.5 get_module_of_link
      • 7.1.6 write_link_clean
      • 7.1.7 read_page_dirt
      • 7.1.8 prepare_tidy_page
      • 7.1.9 prepare_page_name
      • 7.1.10 write_page_clean
      • 7.1.11 wrangle_link_page
    • 7.2 Server Functions
      • 7.2.1 filter_chapter_linkpage
      • 7.2.2 get_pageview
      • 7.2.3 get_page_name
      • 7.2.4 get_unique_page_name
      • 7.2.5 creat_page_table
      • 7.2.6 get_click_per_link
      • 7.2.7 creat_link_table
  • 8 Discussion Forum Overview
    • 8.1 Data Cleaning Pipeline
      • 8.1.1 Hierarchy of the Discussion Forums
      • 8.1.2 Hierarchy Data Integrity Issues
    • 8.2 Visualization Reasoning and Caveats
      • 8.2.1 Filter panel
      • 8.2.2 Barchart
      • 8.2.3 Wordcloud
      • 8.2.4 Threads Table
      • 8.2.5 Searches Table
      • 8.2.6 Interactivity and Navigation
  • 9 Discussion Forum Dashboard Functions
    • 9.1 Wrangling Functions
      • 9.1.1 get_activity_levels
      • 9.1.2 get_discussion_vars_json
      • 9.1.3 get_discussion_vars_xml
      • 9.1.4 infer_post_subcategories
      • 9.1.5 prepare_json
      • 9.1.6 prepare_posts
      • 9.1.7 prepare_searches
      • 9.1.8 prepare_views
      • 9.1.9 prepare_words
      • 9.1.10 prepare_xml
      • 9.1.11 wrangle_forum
    • 9.2 Server Functions
      • 9.2.1 apply_forum_filters
      • 9.2.2 calculate_forum_searches
      • 9.2.3 count_authors
      • 9.2.4 count_posts
      • 9.2.5 count_views
      • 9.2.6 filter_forum_elements
      • 9.2.7 filter_wordcloud_data
      • 9.2.8 gather_post_types
      • 9.2.9 get_author_count
      • 9.2.10 get_forum_threads
      • 9.2.11 get_label_lengths
      • 9.2.12 get_post_types
      • 9.2.13 get_subcategory_options
      • 9.2.14 get_target_word_counts
      • 9.2.15 get_wordcloud_data
      • 9.2.16 make_forum_barplot
      • 9.2.17 make_forum_breakdown_barplot
      • 9.2.18 make_wordcloud
      • 9.2.19 set_axis_limit
      • 9.2.20 set_breakdown
      • 9.2.21 set_forum_plot_title
      • 9.2.22 set_forum_plot_ylabel
      • 9.2.23 set_forum_threads_title
      • 9.2.24 set_wordcloud_title
      • 9.2.25 specify_forum_data_selection
      • 9.2.26 update_forum_data
  • 10 Problem Overview
    • 10.1 Data Cleaning Pipeline
    • 10.2 The Structure of Problems and Assessments
    • 10.3 Visualization Reasoning and Caveats
      • 10.3.1 Module Overview
      • 10.3.2 Hardest Problems
      • 10.3.3 Assessments
  • 11 Problem Dashboard Functions
    • 11.1 Assessment Functions
      • 11.1.1 extract_assessment_json
      • 11.1.2 extract_assessment_csv
      • 11.1.3 join_extracted_assessment_data
      • 11.1.4 summarise_joined_assessment_data
      • 11.1.5 plot_assessment
    • 11.2 Problems Functions
      • 11.2.1 clean_multiple_choice
      • 11.2.2 create_question_lookup_from_json
      • 11.2.3 filter_valid_questions
      • 11.2.4 get_mean_scores
      • 11.2.5 tally_correct_answers
      • 11.2.6 calculate_percent_correct_tbl
      • 11.2.7 get_mean_scores_filterable
      • 11.2.8 join_summary_lookup
      • 11.2.9 summarise_scores_by_chapter
      • 11.2.10 prepare_filterable_problems
      • 11.2.11 get_extreme_summarised_scores
      • 11.2.12 join_problems_to_lookup
      • 11.2.13 join_users_problems
      • 11.2.14 filter_counts
      • 11.2.15 filter_extreme_problem_choices
      • 11.2.16 aggregate_extracted_problems
      • 11.2.17 aggregate_melted_problems
      • 11.2.18 plot_aggregated_problems
      • 11.2.19 plot_problem_chapter_summaries
  • 12 Video Overview
    • 12.1 Data Cleaning Pipeline
    • 12.2 Visualization Reasoning and Caveats:
  • 13 Video Dashboard Functions
    • 13.1 Wrangling Functions
      • 13.1.1 wrangle_video
      • 13.1.2 obtain_raw_video_data
      • 13.1.3 obtain_video_axis_data
      • 13.1.4 write_wrangled_video_data
      • 13.1.5 prepare_video_data
      • 13.1.6 get_start_end_df
      • 13.1.7 get_watched_segments
      • 13.1.8 make_tidy_segments
      • 13.1.9 check_integrity
      • 13.1.10 get_end_time
      • 13.1.11 get_mode
    • 13.2 Server Functions
      • 13.2.1 get_aggregated_df
      • 13.2.2 get_ch_markers
      • 13.2.3 get_video_lengths
      • 13.2.4 get_summary_table
      • 13.2.5 get_video_comparison_plot
      • 13.2.6 get_segment_comparison_plot
      • 13.2.7 get_top_hotspots_plot
      • 13.2.8 get_high_low_plot
      • 13.2.9 get_up_until_plot
      • 13.2.10 get_rank
  • 14 Deployment Walkthrough
    • 14.1 Step-by-step Guide

edXviz: Interactive Visualization of Student Engagement with edX MOOCs

7 Link and Page Dashboard Functions

7.1 Wrangling Functions

7.1.1 read_course_axis

Import course_axis.csv files based on specified course folder name

Description:

     Import course_axis.csv files based on specified course folder name

Usage:

     read_course_axis(input_course)
     
Arguments:

input_course:

Value:

     course_axis

Examples:

     read_course_axis(input_couse = "psyc1")
     

7.1.2 read_link_dirt

Import external_link_dirt.csv files based on specified course folder
name

Description:

     Import external_link_dirt.csv files based on specified course
     folder name

Usage:

     read_link_dirt(input_course)
     
Arguments:

input_course:

Value:

     link_dirt

Examples:

     read_link_dirt(input_couse = "psyc1")
     

7.1.3 hash_username

Anonyanonymize the username column as hash strings in the input_df

Description:

     Anonyanonymize the username column as hash strings in the input_df

Usage:

     hash_username(input_df)
     
Arguments:

input_df: A dataframe containing username column

Value:

     secure_df

Examples:

     hash_username(input_df = link_dat)
     

7.1.4 set_activity_level

Set three levels for students total spending time on course based on
tracklog link data

Description:

     Set three levels for students total spending time on course based
     on tracklog link data

Usage:

     set_activity_level(link_data)
     
Arguments:

link_data: dataframe containing the information of student click any
          links in the edx course

Value:

     link_data

Examples:

     set_activity_level(link_data = read_link_dirt)
     

7.1.5 get_module_of_link

Get which module each external link locate based on link tracklog data
and course axis data

Description:

     Get which module each external link locate based on link tracklog
     data and course axis data

Usage:

     get_module_of_link(link_data, course_axis)
     
Arguments:

link_data: dataframe containing the information of student click any
          links in the edx course

course_axis: dataframe containing all elements in the edx course

Value:

     link_data

Examples:

     get_module_of_link(link_data = set_activity_level, course_axis = read_course_axis)
     

7.1.6 write_link_clean

Export the tidy link dataframe

Description:

     Export the tidy link dataframe

Usage:

     write_link_clean(input_course, cleaned_data)
     
Arguments:

input_course:

cleaned_data: tidy link dataframe

Examples:

     write_link_clean(input_course = "psyc1",cleaned_data = get_module_of_link)
     

7.1.7 read_page_dirt

Import page_dirt.csv files from specified course folder name

Description:

     Import page_dirt.csv files from specified course folder name

Usage:

     read_page_dirt(input_course)
     
Arguments:

input_course:

Value:

     page_dirt

Examples:

     read_page_dirt(input_couse = "psyc1")
     

7.1.8 prepare_tidy_page

Prepare tidy page tracklog data for making page summary table

Description:

     Prepare tidy page tracklog data for making page summary table

Usage:

     prepare_tidy_page(page_data)
     
Arguments:

page_data:

Value:

     log_data

Examples:

     prepare_tidy_page(page_data = read_page_dirt)
     

7.1.9 prepare_page_name

Get non-video and non-problem element name then create a chapter/module
column for all these course elements

Description:

     Get non-video and non-problem element name then create a
     chapter/module column for all these course elements

Usage:

     prepare_page_name(course_axis)
     
Arguments:

course_axis: dataframe containing all course elements information

Value:

     page_name

Examples:

     prepare_page_name(course_axis =  read_course_axis)
     

7.1.10 write_page_clean

Export the tidy page dataframe

Description:

     Export the tidy page dataframe

Usage:

     write_page_clean(input_course, cleaned_data)
     
Arguments:

input_course:

cleaned_data: tidy page dataframe

Examples:

     write_page_clean(input_course = "psyc1",cleaned_data = prepare_tidy_page)
     

7.1.11 wrangle_link_page

Read in three csv files getting from rbq.py : course_axis.csv,
external_link_dirt.csv and page_dirt.csv; perform wrangling; export
three csv files for building link_page_dashabord: external_link.csv,
page.csv and page_name.csv

Description:

     Read in three csv files getting from rbq.py : course_axis.csv,
     external_link_dirt.csv and page_dirt.csv; perform wrangling;
     export three csv files for building link_page_dashabord:
     external_link.csv, page.csv and page_name.csv

Usage:

     wrangle_link_page(input_course)
     
Arguments:

input_course:

Examples:

     wrangle_link_page(input_cours = "psyc1")
     

7.2 Server Functions

7.2.1 filter_chapter_linkpage

Filter course item dataframe by the selected course module.

Description:

     Filter course item dataframe by the selected course module.

Usage:

     filter_chapter_linkpage(input_df, module = "All")
     
Arguments:

input_df: The link dataframe or page dataframe.

  module: One of the modules in the course

Value:

     A dataframe filtered by the selected course module.

Examples:

     filter_chapter(tower_item, "all")
     

7.2.2 get_pageview

Count the pageview of each page.Here, we set a threshold for only
counting pages have been viewed by more than certain amount of
students.

Description:

     Count the pageview of each page.Here, we set a threshold for only
     counting pages have been viewed by more than certain amount of
     students.

Usage:

     get_pageview(filtered_log_df)
     
Arguments:

filtered_log_df: A page related tracklog dataframe after filtering.

Value:

     page_student A page summary dataframe containing the pageview of
     each pages.

Examples:

     get_pageview(filtered_log_df = log_dat)
     

7.2.3 get_page_name

Get the name description of each page.

Description:

     Get the name description of each page.

Usage:

     get_page_name(page_name_df)
     
Arguments:

page_name_df: A dataframe we get after joining page dataframe and page
          name dataframe.

Value:

     each_page

Examples:

     get_page_name(page_name_df = page_name_mapping)
     

7.2.4 get_unique_page_name

Remove duplicated name for each page.

Description:

     Remove duplicated name for each page.

Usage:

     get_unique_page_name(each_page_df)
     
Arguments:

each_page_df:

Value:

     all_pages

Examples:

     get_unique_page_name(each_page_df = each_page)
     

7.2.5 creat_page_table

create a page summary table to show page(name,clickable hyperlink) and
its page view sorted by pageview in descending order

Description:

     create a page summary table to show page(name,clickable hyperlink)
     and its page view sorted by pageview in descending order

Usage:

     creat_page_table(page_summary)
     
Arguments:

page_summary:

Value:

     page_df

Examples:

     creat_page_table(page_summary = reactive_page())
     

7.2.6 get_click_per_link

Compute how many times each external link have been clicked.

Description:

     Compute how many times each external link have been clicked.

Usage:

     get_click_per_link(link_df)
     
Arguments:

 link_df: A link dataframe.

Value:

     link_num A link dataframe with number of click information.

Examples:

     get_click_per_link(link_df = link_dat)
     

7.2.7 creat_link_table

Create a page summary table to show page(name,clickable hyperlink) and
its page view sorted by pageview in descending order

Description:

     Create a page summary table to show page(name,clickable hyperlink)
     and its page view sorted by pageview in descending order

Usage:

     creat_link_table(link_summary)
     
Arguments:

page_summary:

Value:

     page_df

Examples:

     creat_page_table(page_summary = reactive_page())