Project 1 : DSL design, evaluation and implementation

Task Description  

• Register for the project urgently if you have not done so already but plan to participate in Project 1. Please do this by emailing Mayant ( mayantm@cs.ubc.ca ); see project registration.
  • You'll be assigned a TA to mentor your group throughout the project.
  • You will also have a git repository set up for your team to use throughout your project. The git repository will help the course staff judge your progress, and address any group dynamics issues that may come up. 
• Think up a use-case for a new small domain-specific language (DSL) 
  • Make this explicit: what will the DSL enable, and how? Submit your group's idea to your TA for feedback.
• Define the DSL informally on paper: its grammar and semantics
• Divide the project/implementation work into clearly-defined responsibilities
  
  • Consider how to divide the work into separate modules of work, and how you can enable working on these in parallel with each other.
  • Make sure to use version control throughout your project; use the git repository we will provide to you.
• Perform a task-driven user study on the effectiveness of the language early in the project. We’ll see some material on this in Week 3.
• Modify the language based on the results of the study; redesign accordingly.
• Implement the language (some possible techniques over the next lectures).
• Perform a task-driven user study on effectiveness of the complete language. Reflect on the outcome: what is your evaluation?

Make a 3 minute video, capturing your entire process, user studies and their outcome, and evaluating the design and usefulness of your language. You should submit your video to your GitHub, either as a video file in version control (under 2GB), or video link (e.g., to YouTube or a file-hosting service). Please submit videos/links in which your team number is clearly visible (e.g. in the video filename) and use standard file formats.

We'll follow up with some lightweight testing of our own; we might ask you to briefly meet with your TA mentor to assist with this testing.

Grading Guidelines (cf. DSL-specific Medium-level Learning Objectives)

We'll largely use similar criteria to those you're expected to apply yourselves in the DSL part of the course, according to the DSL-related Medium-level Learning Objectives (LOs). In particular, we'll consider:

• The original motivation for your DSL; there should be a clear and convincing use-case (considering both the task and the users it's aimed at) (LO 1).
• The design of your language itself; to what extent does this seem a good fit for the use-case (task and users)? Does it contain 3 complex features that interact with each other? (LO 2).
• The quality of your implementation in terms of a smooth user experience (including e.g. usability, error handling in a reasonable way) (LO 3). Including reasonable documentation outlining how to run your tool, as well as the known limitations, will help in grading.  You're expected to follow good software engineering practice (clear code, organised, comments, version control etc.) although we won't perform code reviews as a rule, unless wanting to check out something specific.
• User evaluation during the project: performing user studies and acting appropriately on your feedback. Summarize the contents of the user studies and the feedback you got somewhere in the final repo (e.g., Checkins.md or Userstudies.md), so it is clear they were performed (LO 4).
• Your own evaluation of your work at various stages, including analysis of the user study results. Identifying some negative points in your design/implementation won't necessarily reflect badly on your grade; instead, if you explain your judgement clearly with respect to the principles taught in the course then this can be a positive point (LOs 5 and 6).
• Team management and organisation: dividing up and coordinating responsibilities within your group, submission of the check-in reports on time, communication with your TAs, for the final submission: the video sticking to the right length (without alarming amounts of speeding up etc.). Please take care to not exceed 3 minutes; you will be docked marks for a too-long video.  We will also consider in the grade whether at least some group member was present at the video fest to answer questions. 

Check-ins and Reports

Project Check-ins are (lightweight) subgoals during the project, used to document your process and get explicit feedback from your TA (of course, you're also welcome to talk with your TA about other things!). Each check-in requires you to communicate information with your TA, get their feedback and add a short corresponding description to your project repositories. By default, you should speak with your TA during their office hours; it might take 15-20 minutes to talk through your check-in. Note that all team members are expected to participate in each discussion with your group's TA; if you have a special circumstance in one week that's no problem, but make sure to let your TA know in advance.

It's your team's responsibility to organise how you'll get your check-in reports signed off; the usual way would be to have a booked slot with your TA once a week. Once you've had TA feedback on the check-in, commit an entry to a CHECKINS.md file in your project git repository including a summary of the relevant information and any relevant notes from your discussion with your TA (this can be a paragraph or two; longer if you would like). The entry should at least briefly summarise the points from the check-in. Make sure to push your commit by the check-in deadline!

Check-in Report deadlines are by Friday 5pm. Note that nothing stops you from discussing and submitting these check-in points early if you'd like to (you could even submit more than one at once if you think you're ready). You should try to keep ahead of these goals in terms of the actual project progress; the check-ins are the latest you should have thought through these points, but not necessarily the recommended times to do these parts of your project work.

See the separate Canvas assignments for full details; below is a summary only.

Check-in 1 (Week 2): Discussion of your planned DSL idea, including the anticipated use-case, user-base, and 3 complex features (as well as a justification for why they are complex. Notes of any important changes/feedback from TA discussion. Any planned follow-up tasks or features still to design.

Check-in 2 (Week 3): Revised proposal of your planned DSL idea. Planned division of main responsibilities between team members, considering how to enable working in parallel as much as possible. Roadmap/timeline(s) for what should be done when, and how you will synchronise/check-in with each other to make sure progress is on-track. Summary of progress so far. Individual surveys on team progress are a separate requirement for everyone (details in a separate assignment). 

Check-in 3 (Week 4): Mockup of concrete language design (as used for your first user study). Notes about first user study. Any changes to original language design, timeline/plan. Revision of project tests.

Check-in 4 (Week 5): Status of implementation. Plans for final user study. Planned timeline for the remaining days. Individual surveys on team progress are a separate requirement for everyone (details in a separate assignment).

Check-in 5 (Week 6): Status of second user study. Last changes to your design, implementation or tests. Plans for final video (possible draft version). Planned timeline for the remaining days.

We hope you enjoy the projects! As usual, if you have any questions, please ask in Piazza or pop along to an office hour.

Best wishes,

Caroline and the TAs