Key Information

The goal of the final course project is to implement some non-trivial deep learning model or component within the needle framework. The assignment as a whole is fairly open-ended, and many different options are available: you could implement some class of model (requiring functionality beyond what we cover in the homework), some new suite of optimizers, some new functionality within the autodiff framework, or some new backend support for needle.

The one underlying component, however, is that you implement your idea within the needle framework itself, not e.g., using TensorFlow or PyTorch.

Component Due date (CMU course)
Project Propsal (+ Groups) 11/4/24
Required Check-in 11/15/24
Presentation videos 12/02/24
Final Report 12/11/24

Possible project suggestions

The following are suggestions for project ideas. You’re welcome to look into these, of course, but we encourage you to think about topics that excite you, and to work on these, rather than just picking solely from the list below. Hopefully, however, these give you some sense of the expected scope of the project.

  • Autodiff support for linear algebra operators like inverses, eigenvalue decompositions, etc, and an illustration of its use within an architecture.
  • Autodiff support for Fourier Transforms, with possible applications to convolutional networks.
  • Autodiff support for other classes of implicit layers beyond the equilibriums models describerd in class, like those used in Neural ODEs.
  • Adding support for sparse matrices/vectors + autodiff, with application e.g. to graph neural networks.
  • Adding acceleration backends for your favorite accelerators, such as an Apple M1 hardware, AMD GPUs via ROCm.
  • Any work in advantaged compilation / hardware optimization of networks, such as fusing operators in computational graphs, etc.

Groups

  1. The final project should be done in groups of 2-3 students.
  2. One student in each group will submit the project proposal and the Andrew IDs of other students in the group.

We will not approve any groups of four or more students. In very rare cases, we may approve students who want to work on a project alone, but there must be a well-founded reason for this beyond not being able to find a group to work on a particular topic of interest.

Proposal

You should submit your proposal and your group via Mugrade by 11:59pm ET, on Nov 4. Only one student in each group needs to submit the proposal, and it will be visible to the rest.

The proposal should be no more than 1000 characters. In it, you should briefly cover:

  1. What topic you will cover in the proposal
  2. What resources (papers, code, etc), you intend to build upon for the project

Please use this form to give us more information about your project.

Project Check-in

Your objective for this project check-in is to show your project mentor that you’re making progress and to also get some feedback on your project plan. Please use this form to give us more information about your project progress. Once we notify you of your project mentor, you may reach out to them to schedule a time to meet. We will notify you once we have forms filled out by each group.

Video presentations

Each group must prepare a 90 second presentation describing their project. We will watch these videos as a group on the final day of class (all students must attend either in person or on Zoom this final day), and the

  • a recording of a slideshow with voiceover,
  • a screen-capture of the system you’ve built,
  • an animation explaining your project, or
  • anything as long as it is a video file and it lasts no more than 90 seconds.

You must submit a link to this video hosted on YouTube. If you’re unable to access YouTube due to firewall restrictions (but only for this reason), you may submit a link to the video file and the TAs will be able to upload.

Final Report + Code

The final report must be submitted as colab notebook. The notebook itself should work contain both prose (about 2000 words) explaining the idea, and the work done in the problem, and then contain a link or a direct download for the code you wrote for your project.

The grading of the project report will be broken down as follows:

  • 50% Prose: The written portion of your report should describe the idea you are implementing clearly and concisely. It should cite relevant background work (either other libraries or papers), and provide an overview of the software architecture and design that informed your system. The written portion should contain approximately 2000 words.

  • 50% Code: Your code should implement the desired functionality correctly and concisely. Unlike the homeworks, we will assign some credit here to the layout and organization of your code, so you should follow standard design principles of commenting code, separate the code into proper class structure, etc. Code should be included as a private downloadable link provided in the notebook, such as a Google Drive link (not a public github repo, since this would release solution code for the needle library).

Grading

Your project will be graded primarily on how well you are able to extend the ideas presented in class to build an interesting and useful extension of the methods we have covered. Your project will be graded based both upon your prose and code, so the code should be written in a legible manner meant to be read, with appropriate structure and comments, etc.

Each project report will be evaluated independently by two members of the TA staff and the instructors. We will provide your final grade along with short feedback on the project.

Plagiarism

Plagiarism, the unattributed copying of ideas, prose, pictures, program code, or any product of human effort, is forbidden. Anything that you include as part of the written code, prose, or figures must be your own work, and cannot be copied from elsewhere (i.e., even with citation). This applies even to figures … the reason we include this clause is that we don’t want people to simply lift all their figures from some other source. If you want to explain something where a figure would be useful, think about how to create a similar figure yourself to explain the concept from scratch.

Breakdown

The score breakdown is:

Part Weight
Proposal 5%
Video 25%
Report + code 70%