The SmartPlanner

Contents

1. Backend Structure
2. Basic Planner Functionalities
3. SmartPlanner Functionalities

Backend Structure

SmartPlanner, like the other SmartUni components, is based on the Django web framework, which is able to manage much of the necessary backend structure out of the box, particularly taking over most aspects of low-level HTTP request handling.

Django provides many pre-made variations for views; of those pre-mades we used Create, Edit and Delete views for events and tasks etc., as well as a generic View class for the Planner pages. This class does nothing but identify the request method (e.g. post or get), then forward the request to the appropriate (developer-provided) function, or raise an error if there is no such method.

In the cases where the functions our team needed were not included in the standard Django toolset, we wrote the views ourselves. Most of these are function-based views because of their greater versatility compared to class-based views (whereas the simpler ones are class-based in order to take advantage of class inheritance). These complex views include, among others, dynamically inserting tasks into the calendar or removing them, to allow drag-and-drop placement.

The models have all their essential functionalities like validation, saving, or deletion handled by Django natively. There were hardly any alterations made to these except for disallowing more than one settings instance per user to exist at a time.

Django does not include pre-made standard HTML templates for rendering the HTTP responses (and we did not obtain any from elsewhere), but the templates we created are quite simple: they either rely on HTML forms, are straightforward text popups, or have most of their functionality contained in attached JavaScript rather than in the HTML file. The calendar overview in particular is purely JavaScript, with the template being only a container.
Features that are needed on multiple pages, like the sidebar, are part of a start page that all other templates derive from. The startpage itself is inaccessible by normal means, as it automatically redirects to the calendar overview.

Whenever data needs to be collected from the client (on a settings page, for example), this is done through HTML forms. Form creation, validation, and similar necessities are handled by Django, except for ensuring the correct processing order of forms that contain other forms, like the event creation containing another form to create a category for the event. In these nested forms, the inner form must be validated and saved before the outer form, and the outer form must not be saved if the inner form is invalid. This requires changes to the validation and request handling methods. The save method, however, can remain unchanged.
The fields contained in the forms, called the widgets, are again standard Django options like text or date fields, except for our custom duration and color widgets. The former new widget is used in task creation, and the latter for category customization.

Basic Planner Functionalities

Tasks and Events

Events and tasks are part of the core functionalities of the SmartPlanner. Events enable the user to create fixed appointments, like going to the doctor, while tasks correspond to more flexible, open to-dos, like doing laundry. Upon creation, events are directly visible in the appointed time slot in the calendar. Tasks, on the other hand, appear in a task sidebar and can be added to the calendar manually via drag and drop or via the Smart Planning function.

Create an Event

Events can be created by choosing the “Event” option of the plus button (Fig. 2, left). A pop-up opens that enables the user to create a new event (Fig. 2, middle). Mandatory fields are the title, the start date, the start time, and the end time. Additional optional fields are the category with its corresponding color, the end date, a location, a comment, and the recurrence pattern option, that creates recurrent events. Only if all mandatory fields are filled in will the submit button in the bottom right corner become active and the event can be created. Upon clicking the active submit button, a new event will be added to the Event database object and the event will appear directly in the calendar. One can cancel the event creation by clicking on the close button in the upper right corner. A warning pop-up (Fig. 2, right) appears that asks the user to confirm the closing of the event creation pop-up.

Fields of the event creation pop-up:

• Title: The title of the new event. It can be freely chosen by the user, but cannot exceed 100 characters.

• Duration: The duration of the event consists of a start and end date at the top and a start and end time at the bottom. By default, start and end date are the current date. Start time and end time are set at 12 pm and 1 pm respectively. Using the arrow keys to the right, the user can change the start and end times by adding an hour and the start and end dates by adding a day. The arrows on the left side enable the opposite. Clicking on the dates opens a small calendar that allows the user to choose a new date in the future, while clicking on the times enables the user to manually insert a new time. The end time and end date can never be set before the start time and start date. However, dates in the past can be selected.

• Category: The category describes a user defined category the event belongs to. By default, the event has no category attached. A category can be added by writing the category name in the text field.

• Location: The location is a short description of where the event takes place, for example ‘University Building E33.’ It cannot exceed 100 characters.

• Comment: The comment enables the user to note down important specifics for their event. For example, ‘Do not forget your laptop.’ It is a text field that allows up to 2000 characters.

• Add recurrence pattern: The recurrence pattern is an additional functionality that allows the user to create recurrent events. It is disabled by default and can be selected by clicking on the recurrence field.

Create a Task

Similar to events, tasks can be created using the plus button by choosing the “Task” option. Clicking on this opens up a pop-up that enables the user to create a new task (Fig. 3). Mandatory fields for a task are the title, deadline, duration, and priority. Optional are adding a category with its corresponding color, a description, and the option to enable smart planning of the task using the AI. If all mandatory fields are filled in, the submit button in the bottom right corner for creating a new task is enabled. Upon clicking on it, a new task will be added to the Task database object and the task will appear in the task sidebar. To stop the creation of a task, the user can exit the pop-up by clicking on the close button in the top right corner. The same pop-up as during canceling event creation appears and asks for confirmation.

Fields of the task creation pop-up:

• Title: The title of a new task can be freely chosen by the user, but cannot exceed 100 characters.

• Deadline: The deadline is the date and time by which the task should be completed. It consists of a date and a time which are preset to the selected date (visible in the URL) and the last full hour. Both can be changed using arrows on the left and on the right of the date and time. Using the arrows on the left subtracts the date by one day and subtracts the time by one hour respectively. The arrows on the right enable the opposite. Neither date nor time can be set to the past. This is ensured by disabling the left arrows. Additionally, by clicking on the date directly, a calendar opens that allows a specific date in the future to be picked. For the deadline time, clicking on the time enables the user to manually set a new time.

• Duration: The duration is the approximated time necessary to complete the task. It is the time the task will later take up in the calendar upon insertion. The duration is preset to 30 minutes but can take on any value between five minutes and 23 hours and 59 minutes. The duration can only be set by using the arrow-buttons atop and below the time. Using the arrow-buttons on top, the hours can be increased in increments of one and the minutes in increments of five. The arrow-buttons at the bottom do the opposite and decrease the duration in increments of five.

• Priority: The priority defines how important the task is to the user. It is a drop down menu that is preset to low and can take on the values low, medium and high.

• Category: The category describes a user defined category the task belongs to. It works for the task just like the category for the event.

• Description: The description allows the user to note down important aspects of the task, for example the topic and requirements of a term paper that has to be written. It is a text field that allows up to 2000 characters.

• Allow Smart Planning: By default, the AllowSmartPlanning checkbox is enabled. This allows the task to be placed in the calendar by the artificial intelligence (AI) component. To disable this feature, the user has to deselect the checkbox.

Detailed View of Tasks and Events

Since the task visualization in the sidebar and the task and event items in the calendar do not show all information, a detailed view for both tasks and events was added (Fig. 4). The detailed view opens once the user clicks on the event in the calendar, or on the task either in the calendar or in the sidebar. It shows all the information inserted upon event or task creation in a new pop-up. To leave the detailed view, one has to click the close button in the upper right corner. Furthermore, the detailed view allows the user to edit and delete the selected event or task via the pen and trash can icon in the upper right-hand corner.

Editing a Task or Event

By clicking on the pen in the detail view of an event or a task, the creation pop-up opens up with the given information filled in. Any of the mandatory and optional fields can be edited. To save the changes, the submit button has to be clicked. Any changes to title and category are directly visible in the calendar and additionally the task sidebar for tasks. In the calendar, changes to the duration are visible as well. The editing process can be stopped without saving the changes just like during the creation process. For this, the close button in the upper right corner has to be clicked.

Deleting a Task or Event

A task or event can be deleted by clicking on the trash icon in the detail view. A pop-up will open asking for confirmation. If the user confirms, the task or event will be deleted from the respective database and will no longer appear in the calendar or the task sidebar.

Task Overview

To enable the insertion of tasks into the calendar, a retractable task sidebar was created. By default, it is closed. Upon opening the sidebar, one either receives a notification that there are no open tasks that can be inserted into the calendar, or is shown all open tasks sorted by priority, as well as up to ten of the most recently completed tasks (Fig. 5). If tasks are overdue, meaning the deadline date and time have passed, a red ‘overdue’ message appears written on the right, beside the title.

Generally, each task in the list is shown as its own entry containing the title and a bar in the color of the category it belongs to. On the left side of the entry, there is a checkbox for ticking off tasks once they are done. By clicking on the checkbox, the task will automatically be moved into the done list. Tasks in the done list have their title crossed out. If the task was mistakenly moved to the done list, it can be moved back into their priority list by unchecking the checkbox again.

To simplify inserting tasks into the calendar, each task entry in the list can be dragged and dropped onto the calendar (Fig. 6, left), both in the weekly and in the daily view. Tasks that are done or are already in the calendar cannot be inserted again. If a task is neither done nor already in the calendar, dragging the task over the calendar makes a draggable calendar entry appear in the calendar. This entry is the size of the task duration and contains its title. If it is dropped in the calendar, it appears at the time it was dragged to last. Tasks in the calendar are draggable as well (Fig. 6, right). This allows repositioning in the calendar and removing the task from the calendar by dragging them onto the sidebar.

Weekly and Daily Overview

A calendar is a temporally structured collection of events, which immediately prompts the question of what the temporal structure is to be. The main question is the timeframe that each overview panel should span — in most human contexts, neither a minute-by-minute nor a century format would be appropriate, but anything from hourly to five year spans would be.

It was settled on almost immediately upon discussing the matter that our calendar would encompass a weekly and a daily overview, but no views larger or smaller than that. There are few use cases in academic planning contexts for anything more fine-grained than daily anyway, as most relevant activities would not need to be planned to the precise minute. Larger overviews are not without use, but we planned to focus on no more than two.

Both overviews, in addition to their original purpose, also carry a sidebar containing all the open tasks of the immediate time period, so that they may be dragged and dropped into the calendar at a convenient time or ticked as done once they are completed. Certainly, the main attraction of the SmartPlanner is the automated, AI-based placement of tasks, but in this way, the user may keep their autonomous choice over their time planning and override the AI whenever desired.

Implementation

At first, we attempted a simple table-based HTML implementation of the calendar from scratch, reasoning that the available calendar plug-ins had too broad a scope and too many features that we would not use. Using a third-party implementation would also cause a difficult and unclear licensing situation, which would be subject to change if SmartUni were ever picked up as an official tool of the university rather than an academic project.
However, the sheer amount of housekeeping tasks a calendar is to do with regard to the positioning of HTML elements, proper display of dates and the like soon became clear to the development team.

After realizing the full scope of an implementation from scratch, the ready-made plugin from fullcalendar.io was chosen, which offered all the features and under-the-hood functionalities SmartPlanner needed from a calendar (and a number beyond those), especially automatic rendering of calendar entries like our tasks and events, a problem we had at the time not even begun to address. The FullCalendar plugin exists in both a free MIT-licensed standard version and a premium version ^{[license details]} , the free license for which SmartUni would be entitled to use as a GPLv3-licensed open-source project, but as the licence of the end product was not known at the time of choosing the plugin, as universities are not covered under FullCalendar’s non-commercial licence and as in any case the standard features sufficed, we decided on the former version.

URL Structure

The URL structure of these overviews started as the base URL of the SmartCalendar plus the name of the overview — this part indeed never changed, as it formed the base of all other SmartCalendar URLs as well — plus a get request for the date to be displayed. The parameter of this request would usually be changed by clicking the “previous” and “next” buttons or by selecting a date in a calendar pop-up when clicking on the current date. It was also possible to directly enter the ISO-formatted date into the date parameter of the URL. The get request broke whenever the overview was relegated to the background of the view port; as it happened this was the case with every other SmartUni URL, be it event creation or deletion or the task detail overviews, because all of them render as overlaying widgets.

The date was therefore removed from the overview URLs, with displaying the correct date now handled by JavaScript, courtesy of the FullCalendar plugin once again anticipating our needs. While this did indeed correct the background rendering, it did not turn out to be a satisfactory solution either, for the difficulty of redirecting back to the correct calendar date after performing any operation involving a form submission, such as (again) event creation, which would necessarily involve a calendar refresh as well to display correctly. It also removed the ability to navigate to a date of the user’s choosing by entering the date into the navigation bar.

Finally, the date was added back to the URLs, this time as a proper part of the URL rather than as a get parameter, and was added to the base of all SmartCalendar URLs as now it could stand in a non-terminal part of the URL address. This finally solved all of the preceding problems without introducing new problems of its own or complicating the eventual solution further.
The single issue remaining might be that the date format of the URL is not in US or in EU format, but rather in the ISO format of YYYY-MM-DD, but since the user is never required to interact with this part, there is little cause for concern or potential for upset on this point.

Categories and Category Filter

The user has the option to create personalized categories which they can assign to their events and tasks. Categories are saved internally as instances of the Category database object. A category consists of a name that can be at most 16 characters long, as well as a color.

Creating Categories

Categories can be created within the event and task creation pop-ups by typing in the desired category name into a text input field and selecting a color (Fig. 9).

Editing Categories

The color of a category can be edited by going into the edit view of an event or a task and selecting a new color. This color change will be adopted for all items that have the respective category attached. Currently, it is not possible to change the name of a category, or to delete it. However, it is possible to attach a different or new category to an event or a task in the respective edit views.

Filtering by Categories

By having categories attached to events and tasks, the user is able to filter for them in the calendar, as well as in the list view in the daily view. In the row above the calendar, we included a button with a funnel icon. When clicked, a text input field opens. The user is able to type in the category they would like to filter for (Fig. 10).

The category that is filtered for is shown in a filter tag next to the SMART button in the calendar options row. Filtering is not limited to a single category; multiple categories can be selected one after another (Fig. 11). If one of the already selected categories should be excluded from the filtering process, the close icon next to their name in the filter tag can be clicked.

Recurrent Events

The recurrence pattern is an additional functionality that allows the user to create recurrent events. This might be relevant for events such as birthdays, weekly recurring meetings, or daily events. Upon clicking on the field, it extends to show the recurrence options (Fig. 12). The user can individually create their recurrence pattern by choosing a recurrence cycle (Fig. 13) of a year, a month, or every X weeks until an end date (Fig. 14). The end date can be changed similarly to the dates of the duration functionality.

Furthermore, if the ‘week’ option is selected, the user can choose on which days of the week the event shall appear during the recurrence cycle. For monthly and yearly recurrences, the specific date is chosen, but no weekday selection is possible. Upon saving, each recurring event is created and added to the calendar (Fig. 15).

When a user clicks on one of the recurring events in the calendar, they see detailed information regarding the event and its recurrence pattern (Fig. 16).

Further, they can delete or edit the recurring event. When they choose to delete one of the recurring events, a pop-up will appear (Fig. 17). There they can decide whether they want to delete only the present event out of the recurrence pattern or all events in the pattern.

Smart Planner Functionalities

Database Structure

The data necessary for training our AI component (the decision tree) is saved in a database object called DecisionTreeData. To enable the Smart Planning process, we additionally use two support database objects, SmartPlanningSettings and TemporarySmartTasks, containing user choices that are necessary to insert tasks into the calendar based on the user preferences (Fig. 18).

The DecisionTreeData database object contains the training data for the decision tree model and consists of information extracted from all tasks that were inserted into the calendar, but lay in the past (completed or not). In the beginning, when there is not yet enough information about the user, this database is supplemented by data collected from the development team. To make sure the database is always up-to-date, it is updated regularly. Once the user accesses the planner, it is checked if 24 hours have passed since the last database update. If that is the case, all tasks newly fulfilling the requirements since the last update will be checked against the database. This is important to make sure each task only appears once in the database. If the respective task cannot be found in the database, it will be added, otherwise it will be skipped.

The two support database objects, the SmartPlanningSettings object and the TemporarySmartTasks object, receive their content from the smart planning pop-up (Fig. 19) and save important information that simplifies the SmartPlanning process (Fig. 20). The SmartPlanningSettings object saves the settings chosen by the user for the current SmartPlanning process. This allows the insertion algorithm to access the timeframe in which tasks shall be inserted, during which hours and on which days in that timeframe. To save the days, we make use of another database object that is only saving weekdays. In that weekday object we save the weekdays selected in the settings pop-up. In the SmartPlanningSettings object we only save the place in the weekdays object, where we saved the weekdays selected in the settings pop-up. Saving these settings allows us to load the settings pop-up the next time using the previous settings. Each time the SmartPlanning process is started, the past settings are overwritten with the current settings.

The TemporarySmartTasks object saves the unique task IDs of each task selected in the SmartPlanning pop-up. This allows the user to choose which tasks should be inserted in the calendar using the SmartPlanning functionality. We only save the IDs since we have a database object which saves all tasks. Therefore, using the IDs in the TemporarySmartTasks object, it is possible to search the task database object and access a task’s respective information. As such, saving only the IDs reduces the needed saving space compared to saving the whole task again. To make sure only current tasks are inserted, the database is cleared each time when starting the SmartPlanning process (Fig. 20). Afterwards, the selected tasks are added.

Decision Tree

Our choice of AI component architecture and function was shaped by two factors:

• Firstly, we could not explicitly tell an AI what a good work-life balance with reasonable breaks looks like — a rule-based AI would reflect the biases of the creators and thus could not possibly apply to all users. If not given rules, the AI could only learn them with sufficient data, which would require intrusive questionnaires or a feedback system that many users would likely forget about using. We decided against implementing either of these systems.

• Secondly, there is no proper dataset to start the AI with. Data gathered from our team members regarding their task planning and completion habits is not generalizable to the users in all cases. Optimal weekly planning is not a standard machine learning problem either, and therefore no standard datasets exist. Because of this, the AI must be extendable, capable of continuous learning, or quick to re-train from scratch once more data from the user concerned is available. Of these options, only the last one is straightforward as it does not require any complex modifications to the AI, only a repetition of the training process.

Under these circumstances, it was decided that the role of the AI component was to predict at which time in the calendar the specified tasks were most likely to be completed, but to leave any further time management to the discretion of the user.

This function would be accomplished via a binary decision tree. Decision trees are not able to learn continuously, nor can they be straightforwardly expanded once generated. They are, however, quick to train as long as they are limited to reasonable depths. The other option would have been an artificial neural network, which is more complex, more difficult to re-train, and significantly more unstable. Training a network twice on the same data is not guaranteed to yield the same results, which is unsuitable for a stable planning system. Moreover, neural networks require a lot of data to train, which as established above, we do not have. In theory, it would be possible to switch to a neural network once enough data is available, but we did not seriously pursue this idea, since it would have taken too much time and we would have been unable to test it during development.

Decision trees also possess the advantage of having less architectural variety than neural networks, which enabled us to use a pre-existing implementation from scikit-learn.

We would have liked to implement an AI component optimizing the recommended break activities and motivations as well. However, due to time constraints we had to consider this as a proposal for future extension.

Training & Training Data

After the characteristics of the AI were thus chosen, the next step was to determine the necessary training data and to train an initial model.

In seeming violation of all principles outlined above, the data for said initial model was extracted from the development team keeping an activity log over the course of two weeks. However, once an actual user starts using the SmartPlanner feature, the re-training data will be enriched with actual user data. This data will quickly crowd out the initial dataset, and therefore the limitations and biases introduced by gathering data from a small, non-representative group do not pose practical problems.

The only viable alternative, as we did not have the resources to have pre-testers whose data we could collect, would have been using random initial training data, which would not lead to better or more generalizable results considering those would not be based on the habits of real people at all.

The features chosen to describe each task in the training dataset were as follows: the weekday on which it starts, start time, duration, priority, proximity to the deadline, whether or not it is of the same category as the preceding task or event, and finally whether or not it was completed yet. All of these were encoded to integer numbers, e.g. by using the number of the weekday or by assigning numbers to certain duration ranges. The features and ranges are also visible in Fig. 22.
The output that the AI would predict is the completion status of the task. In the training examples this was a binary classification (each task marked either done or not done), whereas the final tree would give probabilities for each task to be completed. To keep the tree from giving only binary classifications, it was culled to a maximum of five levels of depth and a minimum of 5% of training samples in each leaf. This also prevents the tree from “overfitting” the training data and being unable to generalize to unseen data.

Calendar Integration

Start day and time were determined by a scheduler separate from the decision tree, which for each of the first 100 open tasks — a number unlikely to be reached — that were sorted by priority and deadline proximity, would pick random slots from all allowed timeslots of the week (i.e. on allowed weekdays within the allowed times). It would get the completion probabilities for each random slot as predicted by the decision tree, then place the task in the most likely slot that had no overlap with other events and tasks, or else try different slots if no fitting ones existed. This would be repeated until either tasks or open slots ran out, or until there had been ten unsuccessful tries.

If any tasks could not be placed, the user would be informed about this via a pop-up.

It was a deliberate choice to use random sampling of the timeslots, as pre-allotted “optimal” times to be checked first would inhibit the learning of the AI, especially in early stages when the available data would not be very robust. Given enough tries, random sampling ought to cover the data space well enough for practical purposes, as there is a finite number of slots per week.

References

[1] Scikit-learn: Machine Learning in Python, Pedregosa et al., JMLR 12, pp. 2825-2830, 2011