Learning Outcomes and Evaluation

This course has goals with respect to the knowledge and research skills.

Evaluation will be with respect to each of the outcomes and based on a level of mastery: general awareness, competency, or mastery.

By the end of the course students will be able to:

  • Critique common ways that social or scientific applications of ML require violating ML algorithm assumptions and ways to mitigate or adapt.

  • Evaluate ML Research papers for their applicability to scientific and social applications of ML.

  • Communicate about ML and its limitations work to varied audiences

  • Apply ML to scientific and social data responsibly

Activities

  • reading and evaluating ML research papers

  • facilitating and participating in class discussions of the papers

  • producing a replication, demo, or illustration of one concept covered for a broader audience

  • completing a project using ML in a scientific or social domain

  • writing a CS conference style (short & concise) final paper on the project

Evaluation

The grading scheme is rooted in achieving the learning outcomes.

Presentations (20%)

Presenting papers and participating in class will contribute to demonstrating a basic awareness at each of learning objective.

Each class session will be evaluated on if you contribute to discussion or not. This includes both asking questions and answering questions.

Each time you present will be evaluated on specification, your presentation should:

  • summarize the key takeaways for the reading(s) in your own words

  • summarize key details for understanding to facilitate the discussion

  • discussion of strengths and weaknesses of the paper & method

  • describe how this paper relates to bigger ideas in the course or your own work

You’ll present 5 times, but must meet specification for at least 3.

When you present you don’t have to have all the answers, you can have open questions.
The goal is that you guide the discussion by doing the above and opening the floor up for questions.

Questions that will help organize your preparation, but may apply variably to different readings:

  • What is the key question that drove the research?

  • What is the main finding?

  • What is the model assumed in the paper?

  • Did they include experimental results? If so:

    • do the experiments support the claims?

    • what additional experiments would help make the result make more sense?

    • how broad are the experiments, are the context-specific or general?

  • Is there an analytical result? if so:

    • do the conditions for the proof make sense?

    • are they realistic?

    • what questions do you have about the proof?

Discussions and Exercises (20%)

For each topic we cover in class, you should engage fully in the class discussion and practice exercises that are provided if applicable.

To demonstrate engagement you must:

  • provide a good faith attempt at any exercises provided

  • contribute to the discussion (comments and questions both count)

Translation mini project (10%)

For this assignment you can choose any topic other than the one your project is for and produce a short demo, illustration, or replication that makes some aspect of the the topic accessible for a broader audience.

For this, you must submit a one paragraph proposal that describes your demo Once that’s approved that it will count, you have two weeks to build your demo or replication. The latest your demo may be submitted is at the same time as your final project.

The proposal (2%) will be graded on specification and may be resubmitted until successful. Your demo proposal must:

  • state the topic from class your demo relates to

  • state the format/medium your demo will take:

    • illustration, replication, interactive visualization, etc

  • describe the target audience (a particular type of scientists, impacted people, software engineers, layperson, etc)

  • describes what your demo will do by answering the relevant questions from the list below:

    • what will a person learn by reading/ using your demo?

    • if it’s interactive what will vary? what will be the inputs?

    • what specific result will you replicate?

  • describe a demo that is an appropriate scope (not too large or too small)

The demo will be graded on specification (6%) and can be revised and resubmitted one time. Your demo must:

  • describe a topic accurately

  • be accessible to the specified topic model

  • meet the description in the proposal

With your demo or after, submit a one paragraph reflection (2%) describing what you learned doing this exercise. The reflection will be graded on completion.

Project (50%)

The final project is a chance to dive deeply into one of the course topics. It has the following timeline. Percentages below are of the total grade.

Date

Milestone

Submission format

grading

2021-02-19

Area Selection

Consultation meeting and general questions

feedback only

2021-03-01

Topic Selection

Objectives and scope of work

completion (5%)

2021-03-15

Proposal

Problem statement, lit review, method

specification, with revisions (10%)

2021-04-02

Checkin

Consultation meeting and prelim result

specification, no revision (5%)

2021-04-13

Rough draft

Draft ready for peers to read

feedback only, per paper specs

2021-04-19,21

Presentation

talk in class

specification (10%)

2021-04-26

revision plan

plan for final revision, minor extensions

feedback only, per paper specs

2021-05-07

final paper

final paper submitted for grading

specification (15%)

2021-05-07

final reflection

final paper submitted for grading

completion (5%)

Proposal Specifications

Submit a 1.5- 2page proposal in the ACM Proceedings format.

Your proposal should include a concise problem statement, a preliminary literature review that situates your project, a description of method(s) you will use to answer your questions in your project, and the expected outcomes of your project.

The proposal will be graded on if it meets the specification or not, but you will be able to revise and resubmit if the first submission does not. To meet specification it must:

  • be the right length

  • be the right format

  • include all sections

  • be written clearly

  • describe the problem, clearly identifying what the specific goals of your project are

  • describe a tractable project

  • summarize relevant literature for the problem context

  • summarize relevant course-related literature for your project

  • describe what you will do in your project

  • describe what the end outcome of your project.

Checkin Specifications

  • scheduled on time

  • at least one dimension of progress from proposal

Presentation Specifications

Your presentation should:

  • include an agenda for the talk

  • describe the problem

  • summarize relevant backround

  • clearly identify what you did

  • describe findings

  • include concluding remarks on reflection/possible extensions

Paper Specifications

Your final paper should include a concise problem statement, a complete literature review that situates your project, a description of method(s) used your project, findings, and a discussion or future work section.

For it to meet specification it must:

  • be the right length

  • be the right format

  • include clearly marked sections indicating the required content

  • be written clearly

  • describe the problem, clearly identifying the specific goals of your project

  • summarize relevant literature for the problem context

  • summarize relevant course-related literature for your project

  • include clear description of what was accomplished

  • include a clear summary of results (may include null results/ failed findings)