Giters

XuYuanchi / nlp-phd-global-equality

A repo for open resources & information for people to succeed in PhD in CS & career in AI / NLP

Geek Repo:Geek Repo

Github PK Tool:Github PK Tool

Resources to Help Global Equality for PhDs in NLP / AI

This repo originates with a wish to promote Global Equality for people who want to do a PhD in NLP, following the idea that mentorship programs are an effective way to fight against segregation, according to The Human Networks (Jackson, 2019). Specifically, we wish people from all over the world and with all types of backgrounds can share the same source of information, so that success will be a reward to those who are determined and hardworking, regardless of external contrainsts.

One non-negligible reason for success is access to information, such as (1) knowing what a PhD in NLP is like, (2) knowing what top grad schools look for when reviewing PhD applications, (3) broadening your horizon of what is good work, (4) knowing how careers in NLP in both academia and industry are like, and many others.

Contributor: Zhijing Jin (PhD student in NLP at Max Planck Institute, co-organizer of the ACL Year-Round Mentorship Program).

You are welcome to be a collaborator, -- you can make an issue/pull request, and I can add you :).

Endorsers of this repo: Prof Rada Mihalcea (University of Michigan). Please add your name here (by a pull request) if you endorse this repo :).

Contents (Actively Updating)

Top Resources

  1. Online ACL Year-Round Mentorship Program: https://acl-mentorship.github.io (You can apply as a mentee, as a mentor, or as a volunteer. For mentees, you will be able to attend monthly zoom Q&A sessions hosted senior researchers in NLP. You will also join a global slack channel, where you can constantly post your questions, and we will collect answers from senior NLP researchers.)

Stage 1. (Non-PhD -> PhD) How to Apply to PhD?

  1. (Marek Rei, Lecturer@Imperial College London) Advice for students doing research projects in ML/NLP (2022). [Suggestions]

  2. (Andrew Kuznetsov, PhD@CMU) CS/HCI PhD Opportunity Tracker from Twitter (Developed in 2021). http://www.andrewkuz.net/hci-opportunities-2022.html

  3. (Prof Philip Guo@UCSD) Finding CS Ph.D. programs to apply to. [Video]

  4. (Tim Dettmers, PhD@UW) Machine Learning PhD Applications — Everything You Need to Know (2018). [Guide]

  5. (Tim Dettmers, PhD@UW) How to Pick Your Grad School (2020). [Guide]

  6. (Prof Mor Harchol-Balter@CMU) Applying to Ph.D. Programs in Computer Science (2014). [Guide]

  7. (Prof Jason Eisner@JHU) Advice for Research Students (last updated: 2021). [List of suggestions]

  8. (CS Rankings) Advice on Applying to Grad School in Computer Science. [Pointers]

  9. (Nelson Liu, PhD@Stanfard) Student Perspectives on Applying to NLP PhD Programs (2019). [Suggestions Based on Surveys]

  10. A Princeton CS Major's Guide to Applying to Graduate School. [List of suggestions]

  11. (John Hewitt, PhD@Stanford) Undergrad to PhD, or not - advice for undergrads interested in research (2018). [Suggestions]

  12. (Kalpesh Krishna, PhD@UMass Amherst) Grad School Resources (2018). [Article] (This list lots of useful pointers!)

  13. (Prof Scott E. Fahlman@CMU) Quora answers on the LTI program at CMU (2017). [Article]

  14. (Albert Webson et al., PhDs@Brown University) Resources for Underrepresented Groups, including Brown's Own Applicant Mentorship Program (2020, but we will keep updating it throughout the 2021 application season.) [List of Resources]

  15. (Roma Patel PhD@Brown, Prof Nathan Schneider@Georgetown University) PhD Application Series of the NLP Highlights Podcast) (2021). [Podcast] （A new series they launched that addresses all aspects of PhD application. Besides, it is just a great podcast in general that talks about recent NLP advances）

  16. (PhD students in NLP) NLP with Friends Online Seminar Series (recordings available) [Seminar]
    (I am not sure where to put it but I find it to be a great way to learn about what others are doing in NLP)

Summer Research Opportunities

  1. Summer research opportunities for Undergrads (2021). [Twitter Thread]
  2. (ETH) ETH Summer Research Fellowship (every summer). [Apply]
  3. (MPI) Summer Research Internship with MPI (CaCTüS) (every summer). [Apply]

List of PhD Programs

  1. (Zhijing Jin, PhD@MPI & ETH) Tips on PhD Applications with Max Planck Institute and/or ETH in AI (2021). [Suggestions]

Specific Suggestions

  1. (Prof Nathan Schneider@Georgetown University) Inside Ph.D. admissions: What readers look for in a Statement of Purpose. [Article]

  2. (Eugene Vinitsky, PhD@UC Berkeley) A Guide to Cold Emailing (2020). [Article]

  3. (Prof Shomir Wilson@Penn State University) Guide for Interacting With Faculty (2018). [Suggestions]

  4. (Prof Shomir Wilson@Penn State University) Reference Letter Procedure. [Suggestions]

  5. (Nelson Liu, PhD@Stanfard) PhD Statement of Purpose. [Article]

  6. (Suchin Gururangan, PhD@University of Washington) Personal Statement Advice. [Article]

Improve Your Proficiency with Tools

  1. (MIT 2020) The Missing Semester of Your CS Education (e.g., master the command-line, ssh into remote machines, use fancy features of version control systems).

Where Do I Get GPU Computing Resources?

  1. Many people use Colab (and its Pro version costs 9.99$ per month)
  2. For more computationally intensive projects, you can apply to AWS with credits for research.
  3. For general computational needs, you can use a Digitalocean droplet (5-10$ per month), and run your ubuntu machine with any service for some time.

Pursuing a Software Engineer Career Path

  1. (Steve Yegge@Google) Get that job at Google (2008). [Article]
  2. (Carlos Bueno) Get that job at Facebook (2021). [Article]
  3. Coding practice from CareerCup. [Link]

Stage 2. (Doing PhD) How to Succeed in PhD?

  1. (Prof Isabelle Augenstein@UCopenhagen) Increasing Well-Being in Academia (2020). [Suggestions]

  2. (Sebastian Ruder@DeepMind) 10 Tips for Research and a PhD (2020) . [Suggestions]

  3. (Maxwell Forbes, PhD@UW) Every PhD Is Different. [Suggestions]

  4. (Prof Mark Dredze@JHU, Prof Hanna M. Wallach@UMass Amherst) How to be a successful PhD student (in computer science (in NLP/ML)). [Suggestions]

  5. (Andrej Karpathy) A Survival Guide to a PhD (2016). [Suggestions]

  6. (Prof Kevin Gimpel@TTIC) Kevin Gimpel's Advice to PhD Students. [Suggestions]

  7. (Prof Marie desJardins@Simmons University) How to Succeed in Graduate School: A Guide for Students and Advisors (1994). [Article] [Part II]

  8. (Prof Eric Gilbert@UMich) Syllabus for Eric’s PhD students (incl. Prof's expectation for PhD students). [syllabus]

  9. (Prof H.T. Kung@Harvard) Useful Thoughts about Research (1987). [Suggestions]

  10. (Prof Phil Agre@UCLA) Networking on the Network: A Guide to Professional Skills for PhD Students (last updated: 2015). [Suggestions]

  11. (Prof Stephen C. Stearns@Yale) Some Modest Advice for Graduate Students. [Article]

  12. (Prof Tao Xie@UIUC) Graduate Student Survival/Success Guide. [Slides]

  13. (Mu Li@Amazon) 博士这五年 (A Chinese article about five years in PhD at CMU). [Article]

  14. (Karl Stratos) A Note to a Prospective Student. [Suggestions]

Internship

  1. (Elman Mansimov, Research Scientist@Amazon) Tips on summer industry research internships in ML (2021). [Suggestions]

What Is Weekly Meeting with Advisors like?

  1. (Prof Jason Eisner@JHU) What do PhD students talk about in their once-a-week meetings with their advisers during their first year? (2015). [Article]

  2. (Brown University) Guide to Meetings with Your Advisor. [Suggestions]

Coming Up with Good Research Ideas

  1. (Prof Jia-Bin Huang@UMaryland) How to come up with research ideas? (2021). [Suggestions]

  2. (John Schulman, co-founder of OpenAI) An Opinionated Guide to ML Research (e.g., horning your taste) (2020). [Suggestions]
    Interesting snippets: "Goal-driven. Develop a vision of some new AI capabilities you’d like to achieve, and solve problems that bring you closer to that goal.", "If you are working on incremental ideas, be aware that their usefulness depends on their complexity.", "Consider how the biggests bursts of impactful work tend to be tightly clustered in a small number of research groups and institutions. That’s not because these people are dramatically smarter than everyone else, it’s because they have a higher density of expertise and perspective, which puts them a little ahead of the rest of the community, and thus they dominate in generating new results.", "Early on in your career, I recommend splitting your time about evenly between textbooks and papers. You should choose a small set of relevant textbooks and theses to gradually work through, and you should also reimplement the models and algorithms from your favorite papers."

  3. (Prof Fei-Fei Li@Stanford) De-Mystifying Good Research and Good Papers (2014). [Suggestions]
    Interesting snippets: "This means publishing papers is NOT about “this has not been published or written before, let me do it”, nor is it about “let me find an arcane little problem that can get me an easy poster”. It’s about “if I do this, I could offer a better solution to this important problem,” or “if I do this, I could add a genuinely new and important piece of knowledge to the field.” You should always conduct research with the goal that it could be directly used by many people (or industry). In other words, your research topic should have many ‘customers’, and your solution would be the one they want to use. A good research project is not about the past (i.e. obtaining a higher performance than the previous N papers). It’s about the future (i.e. inspiring N future papers to follow and cite you, N->\inf)."

How to Read Papers

  1. (Prof Srinivasan Keshav@Cambridge) How to Read a Paper (2007). [Suggestions]

  2. (Prof Jason Eisner@JHU) How to Read a Technical Paper (2009). [Suggestions]

  3. (Prof Emily M. Bender@UW) Critical Reading (2003). [Suggestions]

How to Express Our Ideas: Writing Papers, Visualization, etc

  1. (Prof Jason Eisner@JHU) How to write a paper? (2010). [Suggestions]
  2. (Simon Peyton Jones@Microsoft) How to write a great research paper: Seven simple suggestions (2014). [Slides] [Talk]
  3. (Prof Jennifer Widom@Stanford) Tips for Writing Technical Papers (2006). [Suggestions]
  4. (Prof Shomir Wilson@Penn State University) Guide for Scholarly Writing. [Suggestions]
  5. (Prof Shomir Wilson@Penn State University) Guide for Citations and References [Suggestions]
  6. (Carmine Gallo, a bestselling author) The Storytellers Secret (2016). [Video]
    Takeaways: Writing the Introduction section and giving talks can also be like telling a Hollywood story: the setting (what problem we are solving; how important it is), the villian (how difficult this problem is; how previous work cannot solve it well), and the superhero (what we propose). For giving talks, starting with personal stories (e.g., a story of grandma telling the kid not to drink and persist the right thing leading to the person's life pursuit on social justice) is very helpful to get the audience involved.
  7. (Maxwell Forbes@UW) Figure Creation Tutorial: Making a Figure 1 (2021). [Suggestions]
  8. UI design as a medium of thought: see Michael Nielsen's explanation of why UI is important for science, Bret Victor's work, Miegakure that visualizes a 4D environment.

Reviewing, Publishing, Attending Conferences, Networking

  1. (Prof Yang Liu, Trevor Cohn, Bonnie Webber, and Yulan He) Advice on Reviewing for EMNLP (2020). [Suggestions]

  2. (Dr. Anna Rogers, Prof. Isabelle Augenstein@University of Copenhagen) What Can We Do to Improve Peer Review in NLP? (2020). [Paper]

  3. (Prof Shomir Wilson@Penn State University) Guide for Publishing in Conferences and Attending Them [Suggestions]

  4. (Prof Emily M. Bender@UW) On Using Twitter (2020). [Suggestions]

  5. (Timothy Ferriss, author of 4-Hour Workweek) 5 Tips for E-mailing Busy People (2008). [Suggestions]

Memoir-Like Narratives

  1. (Prof Philip Guo@UCSD) The Ph.D. Grind: A Ph.D. Student Memoir (last updated: 2015). [Video] (For the book, you have to dig deeply, and then you will find the book.)

  2. (Maithra Raghu, PhD from Cornell, Senior Research Scientist@Google Brain) Reflections on my (Machine Learning) PhD Journey (2020). [Article]

  3. (Prof Tianqi Chen@CMU) 陈天奇：机器学习科研的十年 (2019) (A Chinese article about ten years of research in ML). [Article]

  4. (Jean Yang) What My PhD Was Like. [Article]

Excel Your Research

  1. The most important step: (Prof Jason Eisner@JHU) How to Find Research Problems (1997). [Suggestions]

  2. (Christopher Olah, OpenAI) Research Taste Exercises (2021). [Article]

  3. (Prof Richard Hamming, Turing award winner) You and Your Research (How a research journey is like & how do end up with great research) (1995). [Talk] [Transcript] [Transcript2]
    Interesting snippets: "Knowledge and productivity are like compound interest. Given two people of approximately the same ability and one person who works ten percent more than the other, the latter will more than twice outproduce the former. The more you know, the more you learn; the more you learn, the more you can do; the more you can do, the more the opportunity.", "One of the chief tricks is to live a long time!", "I made the resolution that I would never again solve an isolated problem except as characteristic of a class.".

  4. (Prof Stuart Card@Stanford) The PhD Thesis Deconstructed (2016). [Article]
    Interesting snippets: "People basically read your paper to write theirs. Your ideas are more likely to spread if you help out.", "We once had a Nobel Laureate come talk to us about his ideas for how to do research. His surprising number 1 recommendation: Don’t work in an area that doesn’t have good funding.", "Table 1. Seismic scale of impact"

  5. (Sam Altman, CEO of OpenAI) How To Be Successful (2019). [Article]
    Interesting snippets: "I think the biggest competitive advantage in business—either for a company or for an individual’s career—is long-term thinking with a broad view of how different systems in the world are going to come together... In a world where almost no one takes a truly long-term view, the market richly rewards those who do.", "One of the most powerful lessons to learn is that you can figure out what to do in situations that seem to have no solution.", "you also have to be able to convince other people of what you believe.", "My other big sales tip is to show up in person whenever it’s important. When I was first starting out, I was always willing to get on a plane.", "An effective way to build a network is to help people as much as you can.", "One of the best ways to build a network is to develop a reputation for really taking care of the people who work with you.", "The best way to make things that increase rapidly in value is by making things people want at scale."

  6. (Tim Dettmers, PhD@UW) On Creativity in Academia (2019). [Article]

Grad School Fellowships

  1. (List compiled by CMU) Graduate Fellowship Opportunities [link]
  2. CYD Fellowship for Grad Students in Switzerland [link]

Other Books

  1. The craft of Research by Wayne Booth, Greg Colomb and Joseph Williams.

  2. How to write a better thesis by Paul Gruba and David Evans

  3. Helping Doctoral Students to write by Barbara Kamler and Pat Thomson

  4. The unwritten rules of PhD research by Marian Petre and Gordon Rugg

Stage 3. (After PhD -> Industry) How is life as an industry researcher?

  1. (Mu Li@Amazon) 工作五年反思 (A Chinese article about reflections on the five years working in industry). [Article]

Stage 4. (Being a Prof) How to get an academic position? And how to be a good prof?

List of Job Opportunities

  1. Academic Jobs Online https://academicjobsonline.org/ajo

Professors' Sharing in Videos

  1. (Prof Devi Parikh and Prof Dhruv Batra@GaTech) Humans of AI: Stories, Not Stats [YouTube]
  2. (Prof Manuel Rigger@NUS) Getting Academic Positions (GAP) Interviewing Series (2021). [YouTube Interview Videos]

Overall Experience Sharing

  1. (Prof Philip J. Guo@UCSD) Philip’s notes on the tenure-track assistant professor job search (2013). [Suggestions]

Academic job search advice

  1. (Prof Nicolas Papernot@University of Toronto, Prof Elissa M. Redmiles@Max Planck Institute) The academic job search for computer scientists in 10 questions. [Suggestions]

  2. (Prof Caroline Trippel@Stanford) The Academic Job Search: A Memoir (2020). [Suggestions]

  3. (Prof Shomir Wilson@Penn State University) Guide for the Tenure-Track Job Market in Computer/Information Sciences (2018). [Suggestions]

  4. (Prof Michael Ernst@University of Washington) Getting an academic job (2000). [Suggestions]

  5. (Prof Matt Might@University of Alabama at Birmingham) Academic job search advice. [Suggestions]

  6. (Prof Manuel Rigger@NUS) How Did Professors Find Their Jobs? Part 1: Diversity in Experiences (2021). [Suggestions]

  7. (Prof Radhika Nagpal@Harvard) The Awesomest 7-Year Postdoc or: How I Learned to Stop Worrying and Love the Tenure-Track Faculty Life (2013). [Article]

Learning about Different Schools

Preparing the Application Materials

  1. (Prof Jason Eisner@JHU) How to write an academic research statement (when applying for a faculty job) (2017). [Article]

  2. (Prof Jason Eisner@JHU) How to Give a Talk (2015). [Suggestions]

  3. (Prof Manuel Rigger@NUS) Interview Questions for Computer Science Faculty Jobs (2021). [Question List]

  4. (Prof Austin Z. Henley@University of Tennessee) Faculty interview questions I asked and got asked (2018). [Question List]

Starting as a Professor

  1. (Prof Jason Eisner@JHU) Teaching Philosophy. [Article]

  2. (Prof Isabelle Augenstein, Emily M. Bender, Dan Jurafsky, and Yoav Goldberg) Panel Discussions on How to Teach NLP Courses (2021). [Video]

  3. (Prof Emily M. Bender@UW) Balancing Teaching and Research (2015). [Suggestions]

  4. (Prof Shomir Wilson@Penn State University, motivation and usage of writing guides down) Guide to the Advice [Article]

  5. (Prof Randy Pausch@CMU) Time management (2007). [Video] [Slides]

Stage 5. (Whole Career Path) How to live out a life career as an NLP researcher?

  1. (Prof Charles Ling@Western University, Prof Qiang Yang@HKUST)Crafting Your Research Future: A Guide to Successful Master's and Ph.D. Degrees in Science & Engineering. [Book]

  2. (Prof Michael Nielsen, now an individual researcher) Principles of Effective Research (2004). [Article]
    Interesting snippets: "“We are what we repeatedly do. Excellence, then, is not an act but a habit.” Underlying all our habits are models (often unconscious) of how the world works. I’m writing this essay to develop an improved personal model of how to be an effective researcher, a model that can be used as the basis for concrete actions leading to the development of new habits.", "Make sure you’re fit. Look after your health. Spend high quality time with your family. Have fun. These things require a lot of thought and effort to get right.", "Develop a high-quality research environment", "Developing a taste for what’s important: What do you think are the characteristics of important science? What makes one area thrive, while another dies away? What sorts of unifying ideas are the most useful? What have been the most important developments in your field? Why are they important? What were the apparently promising ideas that didn’t pan out? Why didn’t they pan out? You need to be thinking constantly about these issues, both in concrete terms, and also in the abstract, developing both a general feeling for what is important (and what is not), and also some specific beliefs about what is important and what is not in your fields of interest.", "occasionally set time aside to survey the landscape of a field, looking not just for problems, but trying to identify larger patterns. What types of questions do people in the field tend to ask? Can we abstract away patterns in those questions? What other fields might there be links to? What are the few most important problems in the field?"

Massive collaboration can help science

  1. (Prof Timothy Gowers and Michael Nielsen) Massively collaborative mathematics (2009). [Article]

Further Readings: Technical Materials to Improve Your NLP Research Skills

  1. (Prof Jason Eisner@JHU) Technical Tutorials, Notes, and Suggested Reading (last updated: 2018) [Reading list]

Others

  1. (All kinds of career advice for Cryptography researchers) Mentoring Workshop and Videos (2021) [Videos]

Contributions

All types of contributions to this resource list is welcome. Feel free to open a Pull Request.

Contact: Zhijing Jin, PhD in NLP at Max Planck Institute for Intelligent Systems, working on NLP & Causality.

How to Cite This Repo

@misc{resources2021jin,
  author = {Zhijing Jin},
  title = {Resources to Help Global Equality for PhDs in NLP},
  year = {2021},
  publisher = {GitHub},
  journal = {GitHub repository},
  howpublished = {\url{https://github.com/zhijing-jin/nlp-phd-global-equality}}
}

About

A repo for open resources & information for people to succeed in PhD in CS & career in AI / NLP