PhD (11) Machine Learning (4) Research (3) Graduate School (2)

application

1. Keep your head above water: The initial adjustment to graduate school

The graduate application process, like most academic programs, takes a long time. For roughly a year, you are either researching schools, filling out applications, waiting for responses, choosing between schools that accepted you, visiting them, or waiting for enrollment to begin. Because of this delay, as well as the additional effort it took to move across the country during the COVID-19 pandemic (I drove. It took a very long time.), my arrival to campus felt strangely anticlimactic.

However, suddenly things began to take shape. Zoom calls with my advisor quickly transitioned from pleasantries to research goals. Emails originally filled with local hikes and restaurant recommendations now came with half a dozen publications attached for me to read. Compared to undergrad and industry, where full meetings (classes) are spent giving overviews of the topic and general ideas, this felt like being tossed in an ocean of technicality.

Naturally then, my learning curve as a new graduate researcher was steep. When the state of the art in your field has just been published in a conference, tutorial videos or articles explaining the approach are probably unavailable. Moreover, these articles are written for researchers, by researchers. This felt like the wild west of learning: nothing holding back on technicality, and consistent intersection of multiple - sometimes unrelated - disciplines.

2. Become a sponge: Reading for your life

Most of us expect a few jitters when we start something new. I had felt them many times before: new jobs, schools, cities, you name it. However, something about my transition to a PhD student felt different. From the beginning, it was clear my advisor was expecting me to absorb this information independently. No classes were going to be held on my research topic. Moreover, it quickly became clear that this horizon - the edge of human knowledge - was constantly changing.

Thus, I did the only thing I could think of in that moment and started to read everything. The first research publication I read took me over three hours (and 7 more to fully understand), split between reading and looking up concepts in the paper I did not grasp. The next one had some conceptual overlap and went faster, just under two. For each of these articles, I kept a sheet of notes and logged a summary into a spreadsheet.

After a few days, the six publications were done, but I felt more lost than when I had started. These topics left me with more questions than answers. Instead of going back to my advisor right away, I peeled through the references of the first six papers and made a new reading list. Once I had absorbed 5 or 6 of these, I repeated the process, generally moving back in time.

After roughly two weeks of reading, my advisor had a new request: find some of the state of the art Machine Learning methods for my given project, and implement them such that they can be evenly compared. Though I was slightly anxious about finally having something to produce, it now seemed tractable. The terms, concepts, papers, and authors my advisor referenced were now vaguely familiar. At the very least, I knew where to go if I needed to learn more. It was after this Monday afternoon meeting that I began to feel the beginnings of confidence as a student researcher. Despite the daunting nature of academic literature, I had absorbed enough to tread water.

3. Don’t skimp on the soft skills: Research is a team sport

Fast foward three more weeks, and I had a working piece of software that integrated many of the SOTA (state of the art) novelty detection systems for deep learning (henceforth called the “system”, since all that is a mouthful). As this sytem got larger and larger, less time needed to be dedicated to bulding it. However, now people became more curious about this system. What kind of experiments can it run? How do you interpret the results? Would it be possible to add $X$ feature?

From here, more of my time was spent as a project (or even product) manager. Providing different lab members access to the code, explaining experimental results, and giving presentations to our funders quickly absorbed the time that was originally spent developing the system in the first place. To my surprise, I felt the skills I got as a consultant were shining through my PhD research. Militant organization, clear communication, and timely responses to questions are all essential traits as a researcher, more so than I expected.

Instead of walking through code or papers, I would draw up a few PowerPoint Slides for each meeting with my group, sharing it with them afterwards. All of my code was managed through source control, which made adding contributors fairly trivial. Though these seemed like simple tricks, I quickly realized they were worth their weight in gold. Anyone who has had to manually merge a nasty code conflict or painstakingly attempt to clarify a technial presentation knows that these little backstops can save you hours of time.

4. Balance yourself out: Forget (some of) what you learned in undergrad

As a college student, your job is to learn a subject well enough that you can get a good (or even excellent) job in your field of choice upon graduation. How do employers determine the strength of your candidacy? There are many ways - projects, competitions, writing samples - but there is one single indicator that stands above the rest: GPA. GPA isn’t everything. It is a cruel truth that an excellent GPA on its own will get you nothing more than your foot in the door at most prestigious companies. What’s worse, a low GPA makes getting a competitive job exponentially more difficult directly out of school.

Suffice it to say, GPA is important. The saying “student first, [insert other focus/sport] second” exists for a reason. The majority of your tuition costs go towards classes. Maximizing your grades in these classes signals to employers that you are tenacious and willing to put in the effort, one of the most important traits a person can have.

However, as a PhD student you are no longer measured by your GPA, but by your research output and expertise within a field. This goal is not entirely orthogonal to courses, but they are not completely aligned either. You do not just develop world class expertise overnight, and classes tend to be a fairly efficient means of learning for most people. With that said, classes alone will not give you world-class expertise, and some courses (or professors) will teach you more than others.

I personally fell into this trap when I started my coursework. As an undergrad I placed a very high priority on courses. Even if it was half a letter grade difference, I would put in that extra time. As a PhD student, I started to do the same, and quickly fell behind on my research. Over the last month, I have begun shifting my focus to better balance these two objectives, but frankly I have a ways to go. Changing mindsets after 16+ years of focusing on grades is easier said than done.

5. Keep time for you

A PhD is a grind, no two ways about it. It is also a marathon. Tearing out of the gate and pulling all nighters to get ahead is a losing strategy. Don’t be afraid to challenge yourself and little and step out of your comort zone… just don’t step too far. Everyone has a story of someone burning out after a few months of pushing too hard, be it research, classes, or a job. Every week (or even better, every day), dedicate some time to keeping yourself balanced and happy. Make this non-negotiable. Deadlines or not, take a half hour to do something that helps you decompress.

For the remaining 23.5 hours of the day, my recommendation based on the past three months is listed below.

When I first started my work as a PhD student, some days I would finish work and be completely drained. Physically, mentally, the work felt exhausting. I had doubts, but also remembered that most things get easier with practice and time. Be deliberate with practicing efficiency, but also patient with yourself. No one is perfect.

6. Make friends with failure

You will fail as a PhD student, and fail hard. The thing about research is no one has ever tried the things you are trying. There may be little or no precedent for your experiments and their probability for success. What is important in these moments is to not take the experience of failure personally. Show grace in the experience by clearly stating the logic behind your experimental setup and expected results.

My first project deadline as a PhD student is today. For months, I have been learning about, building, re-building, tuning, and presenting experiments in a particular subfield. The grant I am working on has been set up as a competition as well as a collaboration with several other groups. There are many moving parts to this project whose objective has little or not precedent in the current literature.

Heading into this evaluation, what I built was showing modest performance. Our collaborators then sent over what would be the final dataset. All of a sudden, the performance on our system plummeted. Random chance plummeted. Immediately, we all went scrambling around looking for code bugs. Surely our solution was not that bad. Days passed; we found nothing that would fix the issue. For a week I had tried to determine (and explain to my lab) reasons for this egregious failure. Perhaps I did not choose the correct approach for this type of data. Maybe the model is sensitive to certain types of information. It stung to see something I worked so hard on do so poorly. While I was sulking about this one afternoon my girlfriend reminded me how much I learned from this project. Even if everything was a failure, I still learned a ton about reading academic literature, producing and debugging deep learning models, and much more.

Only a few days before the submission, we decided to check the dataset to see if there were any abnormalities. Our collaborators had created this part of the project and we generally felt confident their work was correct. Shockingly, I noticed that the dataset had been inadvertantly shuffled. The terrible performance had nothing to do with our implementation. Frantically, we worked with our collaborators to fix this error and got the project back on track. This taught me a valuable lesson: No matter what you are doing, as a PhD student it is your job to understand how everything works. Take ownership of your work from the ground up.

As a PhD student, your job is to learn how everything works

Coming up: Money, time, and happiness - Balancing life as a PhD Student

The previous section poses an interesting, more general question: How do you balance your life as a PhD student? Friends, family, travel, exercise, food, finances… there are so many things to spend your time and money on. What balance works for you is likely unique, but my next post will examine some general tips I discovered that help me stay productive while attempting to live a balanced life.