Symphony: Co-Annotation for Hybrid Learning | NTU COOL

Key Summary

Symphony is a feature in NTU COOL that allows teachers and students to annotate text online and discuss annotations in comment threads collaboratively. After joining the NTU COOL team, I was responsible for improving Symphony's user experience, including enhancing the course enrollment process and annotation workflow and redesigning the comment thread layout and information architecture.

These designs made interactions more intuitive and convenient. The revamped version of Symphony was launched in July 2020, and has helped hundreds of teachers facilitate engaging hybrid learning during the pandemic.

Background

NTU COOL is a digital learning platform customized and developed by National Taiwan University's Digital Learning Center, based on the service of Canvas, a renowned international LMS developer.

NTU COOL was initially introduced to supplement the shortcomings of Ceiba, National Taiwan University's original digital learning platform. However, considering factors such as Ceiba's difficult maintenance, limited functionality, and defects, it was decided in 2019 to transition all NTU faculty and students to NTU COOL within 5 years.

However, as users of Ceiba for over a decade, many professors experienced a steep learning curve when switching to NTU COOL. As a result, only less than 10% of courses adopted it, with only 1% of those adopters using the Symphony feature. By improving Symphony's user experience, we hope this distinctive feature will help attract more teachers to adopt NTU COOL as their digital learning platform for courses.

Design Challenge

Since NTU COOL is an improved platform based on Canvas, information architecture and operational processes for many basic functions cannot be modified. Symphony is a plugin developed by NTU COOL, with a relatively unrestricted operational design for its functionality. Still, after integration with NTU COOL, it faces issues of poor findability and difficulty in improvement.

Product Research

Heuristic Evaluation

As this team has two designers and other interns who had previously conducted a heuristic evaluation, I performed another evaluation without duplicating previous results, and identified 11 usability issues. Some of these issues were directly addressed by designers for improvement; some were bugs that needed fixing, while others were incorporated into future design plans for Symphony.

Comparative Analysis

As the design primarily focused on discussion threads, I identified 5 products with similar functionalities in the market based on "collaboration" and "text annotation": XODO PDF Reader & Annotator, Kami, PDFfiller, Google Docs, and Dropbox Paper, to conduct functional analysis and comparison.

According to the survey, besides the mainstream "card-style" annotation presentation, in terms of detailed interactions, reply designs all use a single-level approach. For multiple replies, most products choose to collapse all reply content directly, while a few designs create a scrollable area within the card and limit the overall card height. In Symphony, both designs can be seen simultaneously, but there's no explanation for why different designs are used in different scenarios. Therefore, this survey established three directions for Symphony's card interaction:

1. Adopt a card-style design with single-level replies
2. When needing to reply to specific comments, use account tagging
3. For multiple replies, consistently collapse the content directly

Feature Design

In the design phase, I mainly focused on creating two versions to address the key issue that arose after switching to a card-style layout: "Should comments and questions be placed in different tabs?" I also worked on improving the annotation process and the toolbar interface.

Annotation Card Field Design

Previously in Symphony, there was a "title-only" tab that allowed users to browse all questions and comments at once. However, after changing to a card-style presentation for questions and comments, we also eliminated the title design. Not only do mainstream card-style designs typically not include a title field, but we also found that people didn't consider the title to be a useful design element. Some would write the entire question in the title, while others would just input random text or symbols.

Version One: Separating Questions and Comments

In Symphony's original design logic, these two were already meant to be browsed separately, though they could be viewed simultaneously under the title tab. Logically, these two are not the same type of content, so I believed that keeping them separate was a reasonable design and didn't plan to change it much.

Version Two: Merging Questions and Comments into a Discussion Thread

Although the design of version one was logical, logic doesn't always dictate how users will interact with a system. During team discussions, we kept considering questions like "What if someone can't distinguish between comments and questions?" and "Might some people want to view comments and questions together?"

As a result, we decided to design another version that integrated questions and comments under a single tab called "Discussion Thread". Since we had no basis for determining which design was better or more aligned with user needs, I decided to gather feedback from potential users by conducting simple tests with design prototypes.

UX Testing

I recruited two students from NTU's Department of Library and Information Science. These students have extensive needs for reading texts and highly value classroom interaction and exchanges with classmates, thus fitting our user profile.

In this test, we first asked the participants to operate the version that was currently online, then alternately test the two versions of the design mentioned above. The two participants tested the versions in reverse order to minimize the impact of the operation sequence.

After the testing, I made the following discoveries: Regardless of the version, both were easier to understand and operate than the version currently online.

Separating questions and comments essentially "creates a filter," which is more convenient for those who need such filtering for questions and comments. The advantage of merging them into a discussion thread is that, although the content types are different, they may be related due to context, leading to a more comprehensive understanding.

Therefore, if we could use the discussion thread format and increase the flexibility of filtering, we could benefit from both approaches. After receiving user feedback, we decided to proceed with the discussion thread format for subsequent designs.

Annotation Process and Toolbar Design

After deciding on the card style, we began to explore how to improve the originally confusing annotation process for adding discussions.

The original annotation input field included a title and body, with text formatting options available for the body. Questions and anonymity were set using checkboxes, and after clicking save, they would appear in the "Discussion List".

The design issues to be addressed here include:

1. In response to the card-style discussion thread, the number of input fields, their positions, and text setting options will change
2. Comments and questions should be on the same level, but at what stage should the choice be made?

Design Results for Annotation Process

1. Remove the floating input field, directly add cards in a pending text input state to the discussion thread, and add micro-interactions to guide users to input content in that card
2. Remove text formatting options, but retain the ability to insert LaTeX and images, making the design more suitable for academic use cases
3. Keep the option to set as a comment or question after annotation rather than on the card itself, and present it as buttons at the same level; since annotation is not a common interaction trigger, clicking the "Set as Comment" or "Set as Question" button gives users a better mental expectation that "other interactions are about to occur"

Design Results for Toolbar

When annotating, in addition to text, Symphony also provides "area" annotation to accommodate scenarios where users want to discuss images. However, the previous toolbar design not only made it difficult to visually discover this feature but also failed to indicate which annotation tool was currently selected.

The position of the toolbar is also a design factor to consider. In the old version, the toolbar was placed above the article and expanded by default. However, some teachers felt that this occupied space that could be used for reading the document, which became a more serious issue when needing to enlarge the document for reading.

Therefore, the new version of the design exhibits the following characteristics:

1. Create a visual hierarchy to help users understand that annotation is divided into two modes, and that area annotation is used for image scenarios
2. Design the toolbar to be collapsible, expanding vertically when opened, to minimize space occupation in both states

Statistical Data Design

Finally, in response to changes in all UI components, the statistical data page also needs to be adjusted. Teachers have three main purposes when reviewing data:

1. Check the status of answered questions
2. View the level of discussion activity for the text
3. See which users are participating in discussions

Therefore, in addition to the three original sections, I also adjusted the presented data to "Total annotations," "Number of unanswered questions," and "Total responses," making the displayed data more aligned with teachers' needs.

Future Design Direction

As the future primary teaching platform for National Taiwan University, NTU COOL will inevitably face changes in educational trends, moving towards a role that supports more personalized learning. In this context, teachers will need more data to assist in managing courses and interacting with students. Therefore, for the statistical data section, I have designed more potential data panels and configuration methods for the future NTU COOL, hoping that it will become a valuable aid for teachers and make students' learning more effective.

中文版：重新設計 Symphony，提升混成教學互動率

Symphony 是 NTU COOL 中的一個功能，能夠讓師生在線上協同註記文本，並在討論串中針對註記進行討論。我在加入 NTU COOL 團隊後，便負責改善 Symphony 的使用體驗，包含改善加入課程流程、註記流程，重新設計討論串陳列樣式與資訊架構，這些設計讓互動變得更直觀方便。此設計目前已於 2020 年 7 月上線，截至 2020 年底，Symphony 使用者已達。

背景介紹

NTU COOL 是由國外知名的 LMS 開發商 Canvas 的服務，經國立台灣大學數位學習中心客製化開發後的數位學習平台。NTU COOL 一開始僅是為了補足原本台大的數位學習平台 Ceiba 之不足而引入的系統，然而考量 Ceiba 維護不易、功能有所限制與缺陷等等的條件後，於 2019 年決定在 5 年內讓台大師生全面改用 NTU COOL。

然而作為 Ceiba 十多年的使用者，許多教授在轉換至 NTU COOL 的時候感受到了極高的學習門檻，因此採用的課程僅不到總數的 10%，其中採用者有使用 Symphony 功能的人，僅佔總課程數的 1%。透過改善 Symphony 的使用體驗，我們希望此特色功能能夠幫助吸引更多老師採用 NTU COOL 作為課程的數位學習平台。

設計挑戰

由於 NTU COOL 是基於 Canvas 進行改善的平台，因此許多基礎功能的資訊架構與操作流程都無法更動。Symphony 是由 NTU COOL 開發的外掛程式，功能本身的操作設計較無限制，但在與 NTU COOL 整合後，有可尋性差的問題，且難以改善。

產品體驗研究與分析

Heuristic Evaluation

由於此團隊有兩位設計師與其他實習生，之前已經做過一次的經驗法則評估法，因此我在不與之前評估結果重複的條件下再次進行了評估，並提出 11 個易用性問題。其中部分問題直接由設計師接手進行改善，部分是 bug 需要修復，而部分則是併入未來 Symphony 的設計規劃當中。

Comparative Analysis

由於主要是針對討論串進行設計，我以「協作」和「文本標註」為方向找出市面上 5 個具備相近功能的產品：XODO PDF Reader & Annotator、Kami、PDFfiller、Google Doc 與 Dropbox Paper 進行功能分析與比較

根據調查發現，除了主流呈現標註的樣式是「卡片式」以外。在細節的互動中，回覆的設計皆以單階層的方式進行。在多則回覆的情況下，多數產品選擇直接把所有回覆的內容收合，少數的設計則是在卡片裡再設計一個可滾動的區域，並限制整體卡片的高度，而在 Symphony 中可以同時看見兩種設計，卻沒有辦法說明為何會在不同情境下採用不同設計。因此這項調查確立了 Symphony 在卡片互動上的三點方向：

1. 採用卡片式設計卡片使用單階層回覆
2. 需要針對特定留言回覆時，可使用標記帳號的方式
3. 多則回覆時統一直接將內容收合

功能設計

在設計階段，我主要針對調整為卡片式之後帶來的主要問題「評論與提問是否應該放在不同的頁籤」進行兩個版本的設計，並針對註記的流程以及工具列介面進行改善。

註記卡片欄位設計

以往在 Symphony 會有一個「只呈現標題」的頁籤，讓人一次瀏覽所有的提問與評論。然而改以卡片樣式呈現提問與評論後，連帶的也取消了標題的設計，除了主流卡片式設計皆不會出現標題欄位以外，我們也發現大家並不覺得標題是個有用的設計，有人會直接把問題寫在標題裡，也有人就只是隨意打個文字或符號。

版本一：提問與評論拆開

在原本 Symphony 的設計邏輯中，這兩者本來就是拆開瀏覽的，只是在標題頁籤下可以同時看見兩種內容。在邏輯上，這兩者本來就不是同類型的內容，因此我認為拆開放是合理的設計，並不打算更動太多。

版本二：提問與評論合併成討論串

雖然版本一的設計合理，然而邏輯對不代表使用者會這樣使用，我與團隊在討論的時候，一直不斷思考，「如果有人分不清楚評論或提問呢」、「會不會就是有人想一次看評論或提問」等問題，因此決定再設計一個版本是整合提問與評論於同一個叫做「討論串」的頁籤之下的。由於我們沒有任何依據可以說明哪個設計比較好或是符合使用者需求，因此我決定聽聽潛在使用者的想法，拿設計原型進行簡單的測試。

使用者體驗測試

我總共找了兩位就讀台大圖資所的同學，圖資所的同學都有大量閱讀文本的需求，也非常注重課堂上與同學之間的互動交流，因此符合我們對使用者的設定。這次的測試在一開始會請受測者操作當時上線的版本，在操作後輪流再輪流測試上面兩個版本的設計。兩位受測者先測試的版本順序相反，以減少操作順序造成的影響。

在經過測試之後，我得到了以下發現：無論是哪個版本，都比原本上線的版本易於理解如何操作。提問與評論拆開，其實是「做了一次篩選」，對於需要這種篩選的人而言比較方便提問與評論，而合併成討論串的優點是，雖然內容類型不同，但可能因為上下文有關，因而產生關係，讓理解更全面。

因此，如果能夠使用討論串的形式，並且增加篩選的自由度，就能兼顧兩種方案的好處。在獲得使用者的回饋之後，我們決定以討論串的形式進行後續設計。

註記流程與工具列設計

在決定卡片的樣式後，近一步的我們開始探討如何改善原本令人困惑的註記以新增討論的流程。

原本的註記輸入欄位，包含標題與內文，且內文的格式允許字體上的設定。提問以及匿名皆採用勾選設定的方式，點擊儲存後，就會出現在「討論列表」中。

此處的設計要處理的議題包含：

1. 因應卡片式討論串，內容輸入的欄位數量、位置與文字設定選項會有所變動
2. 評論與提問的階層應該相同，該在什麼階段選擇？

註記流程設計結果

1. 取消浮窗的輸入欄位，直接新增待輸入文字狀態的卡片於討論串中，並增加微互動引導使用者至該卡片輸入內容
2. 取消文字格式設定選項，但保留可以插入 LaTex 與圖片的功能，讓設計更加符合學術使用情境
3. 將設為評論或提問的選擇保留在畫記之後而非於卡片上設定，且以按鈕呈現並為相同階層；因為畫記並非一個常見的互動觸發方式，因此透過點擊「設為評論」或「設為提問」按鈕，使用者更能有「其他互動即將產生」的心理預期

工具列設計結果

在畫記的時候，除了文字以外，Symphony 也提供「區域」的畫記以因應想針對圖片討論的情境。然而以往的工具列設計除了在視覺外觀上難以讓人發現這項功能，也無法讓人知道目前選擇的畫記工具是哪一種。

工具列的位置也是要考量的設計因素，舊版的工具列放在文章上方並且預設展開，然而部分老師認為這有佔據了文獻可閱讀的空間，在需要放大文獻閱讀的情況下，這個問題更加嚴重。

因此在新版的設計展現了以下特點：

1. 透過視覺創造出階層性，讓使用者理解畫記分為兩種模式，且區域畫記是用於圖片情境
2. 將工具列設計為可收合的樣式，展開後為垂直展開，讓兩種狀態都盡可能減少佔據版面的機會

統計數據設計

最後，因應所有 UI 元件的更動，統計數據的頁面也需要有所調整，老師檢視數據有三個主要目的：

1. 查看問題被回答的狀況
2. 查看文本討論熱度
3. 查看參與討論的使用者

因此除了原本就有的三個區塊以外，我也將原本呈現的數據調整為「註記總數」、「未解答題數」與「回應總數」，讓呈現的數據更符合老師需求。

未來設計方向

NTU COOL 作為未來國立臺灣大學的主要教學平台，勢必會面臨教學趨勢變動，邁向更支持個人化學習的角色。老師在這個情境下，經營課程以及與學生的互動就會需要更多的數據來輔助，因此在統計數據的部分，我為未來的 NTU COOL 設計了更多可能會運用到的數據面板與設定方式，期許在未來能夠成為老師的助力，也讓學生的學習更有成效。