Important Dates


Proposal due: Mon 11/4/2024

Milestone report due: Tue, 11/26/24

Final report due: Mon 12/16/24

Grading


Proposal: 10%

Milestone report: 20%

Final report: 70%

Policies

Projects must be undertaken in groups of 3 or 4 people. Individual projects are not permitted, and project with less than three people must seek approval. All projects must have an empirical component.

Report, Milestone, and Proposal Formats: We adhere to the NeurIPS format. You must use the NeurIPS LaTeX format.

Proposal: Your proposal should be a maximum of 2 pages (in the NeurIPS format) and must state your project title and team members. It should also include preliminary potential formulations, a timeline detailing steps for the milestone and project completion, a brief discussion on the algorithms you plan to explore, along with your implementation strategy, and a description of the data (if any) you intend to use, and . Ensure your timeline aligns with understanding the algorithmic approaches.

Milestone Report: For your milestone submission, please prepare a document of up to 3 pages. You are welcome to incorporate any relevant material from this milestone into your final report. In your milestone, it's essential to rearticulate your problem formulation, reflecting its current status. This should include a clear outline of what aspects are under investigation, your motivation for choosing the problem, and any related work if applicable. Also, provide an overview of your preliminary code development or any experiments conducted to date. Particular emphasis should be placed on demonstrating that you have thought through how your project will explore some foundation model component.

Final Report: Your final report should be a maximum of 9 pages, excluding references. It will be evaluated based on the following criteria:

  • Merit: Do you have sound reasoning for the approach? Is the question well motivated and are you taking a justifiably simple approach or, if you are choosing a more complicated method, do you have sound reasoning for doing this?
  • Correctness and Baselines: Did you test your approach on "toy" problems? Is there a natural baseline comparison if your approach is meant to improve on a previous approach?
  • Technical depth: How technically challenging was what you did? Did you use a package or write your own code? It is fine if you use a package, though this means other aspects of your project must be more ambitious.
  • Presentation: Does your report comprehensively explain your methodology, results, and interpretations? Did you incorporate effective graphs and visualizations? How clear is your writing? Have you justified your chosen approach?

Project Ideas

We provide a few project ideas. Please remember that all reports must have an empirical component. Project topics could be any of the following topics or those that build of topics discussed in class, including Parallelization (FSDP, Tensor/Pipeline parallelization), Flash Attention, Sharding (and Jax), diffusion/generative AI models, sampling (diffusion approaches), inference, evaluation.