As I wrap up my first PhD project on retrospection, time has come for a first year retro.

This covers things happened between October 30, 2023 and November 17, 2024. In 2023, I moved from Toronto, ON, to Ithaca, NY, then Palo Alto, CA, and finally settling down in NYC, NY in January 2024. So far I do not regret the decision to quit and move.

Highlights

• Wrapped up months of human-model live deployment on MTurk.
• Completed the first draft 2 weeks ahead.
• Faster at writing. It now takes only one weekend to write a 2 page proposal.
• Improved engineering skills. Things often turned out way easier than I imagined.
• Occasional dopamine rush from random ideas and a delusion of absolute research freedom.
• Reached the top of Koko Crater Arch Trail in Honolulu.
• Wore the Sorting Hat in Museum of Pop Culture in Seattle.

Lowlights

• Dealt with two annoying MTurk workers.
• More comfortable with (silent) rejections. This is perhaps a highlight.
• Communication failure that led to a good amount of stress.
• Social energy drained in too many occasions, e.g., COLM 2024, NLP retreat, student lunches.
• Did not manage to use all of the TPUs.
• The election.

Gradients

Should-haves

• Asked for reference letters weeks in advance, not last minute.
• Taken advice from those more experienced with reviewers and ACs.

On research (They are subjective and need more gradient updates of less bias.)

• Academia is more stochastic on a micro level (advice and reviews for specific projects), but way more predictable as collective groups (research paradigm and popular benchmarks follows 80/20 rules).
• I tend to give the higher scores among all reviewers.
• More words than I thought are devoted to drawing, clarifying, and justifying the scope. The trick of many papers is “just” moving around the conceptual or implementation interface across components.
• Communication and writing are more important than I thought. Good writing is audience-aware and information dense.
• Ease-of-use is more important over other virtuous bits than I thought, i.e., pip-installable, tweet-able gifs, catchy titles, first-to-market, extensibility.
• The benchmark methodology is more pervasive than I thought.
• An average ML reviewer
  • put in less attention than I thought.
  • wants a method diagram, a table, a plot going up. The more benchmark/method names they are familiar with (regardless of relevance), the better.
  • has a broader definition of comparable scope than the authors (which could be a good thing in retrospect).
  • assumes they understand a paper more than they actually does.

What’s next

I think about

• Latent/fluid abstraction synthesis without programs.
• Cooperative learning that seeks to build a common ground.
• What good research is, what valuable research is, and what kind of research I prefer.
• Is everything, even abstract concepts, a remix of a careful selection of everything else? If yes, then the argument against AI for inability to create doesn’t hold.

I want to learn about

• ML theory: learning theory, RL theory, …
• Humanity in general: psychology, sociology, economics, history, …

I want to practice

• Being aware of task rationale, i.e., why am I doing this, and task execution, i.e., what do I need right now.
• Being aware of ego, mine or others.
• Active listening and be kind(er).