joaocarvalhoopen / How_to_become_dangerous_in_algorithms

A guide to the intrepid adventurer.

How to become dangerous in Algorithms

A guide to the intrepid adventurer.

Description

Algorithms and data structures are the bases of all programs, they can be the difference between a program running for 1 seconds vs running for the age of the universe. They can be the difference between finding a rapid solutions by remembering a similar solution, that you can frame a problem with a known problem, or detecting that it is a NP Complete problem because you can frame it as a similar known problem. In that case maybe a simple greedy algorithm is the best choice. When I say “become dangerous in algorithms” I don’t mean to do bad stuff, I mean become really good at algorithms :-D

Algorithms and data structures

Algorithms and data structures can be a tough subject, full of heavy math analysis, but for a gentle introduction that will let you go very far in terms of taking good choices for the most common problems read the following good book. It relies on more that 400 drawing to pass the intuition about algorithms. It uses Python. You can read it easily in one day.

1. Grokking Algorithms: An Illustrated Guide for Programmers and Other Curious People
   by Aditya Bhargava

Then after reading it, you can continue in the easy path, but with more depth and more algorithms and data structures, also in Python.

2. A Common-Sense Guide to Data Structures and Algorithms, 2th Ed
   by Jay Wengrow

Then if you liked the subject and desire to go farther and want to see were the rabbit hole goes, you can read the following four wonderful books, the first one is accessible but at the same time in depth, with a Hitchhiker’s Guide to Algorithms. The following two, are from competitive programming in C++ and the last one the classical bible of algorithms and data structures. In the site of the first one you can see links to good implementation of those complex algorithms shown in the book.

3. The Algorithm Design Manual, 3rd Ed
   by Steven S. Skiena
   https://www.algorist.com/
   The Stony Brook Algorithm Repository
   https://www.algorist.com/algorist.html
   Lectures and slides
   https://www3.cs.stonybrook.edu/~skiena/373/videos/

4. Guide to Competitive Programming: Learning and Improving Algorithms Through Contests 2th Ed
   by Antti Laaksonen

5. Algorithms: For Competitive Programming
   by David Esparza Alba, Juan Antonio Ruiz Leal

6. Introduction to Algorithms, 4rd Ed
   by Cormen, Leiserson, Rivest, Stein

7. Wikipedia list of algorithms
   https://en.wikipedia.org/wiki/List_of_algorithms

8. Algorithm Wiki
   Crowdsourced Resource on Algorithms and their Development
   http://algorithm-wiki.org/

9. The Algorithms - Python
   For a really complete list of implementations of algorithms
   https://github.com/TheAlgorithms/Python/blob/master/DIRECTORY.md

10. The Algorithms - Rust
    https://github.com/TheAlgorithms/Rust

Then if you want to see the life’s work of a great men, one of his masterpieces, Prof. Donald Knuth and appreciate the most beautiful algorithms explained with an incredible attention to the smallest details, see the following book series. It will take several years to go through them. The algorithms are expressed in a assembly instruction set, MMIX, defined explicitly for the book.

11. The Art of Computer Programming, Volumes 1-4A Boxed Set
    by Donald Knuth, Donald John Fuller

12. MMIX Supplement, The: Supplement to The Art of Computer Programming Volumes 1, 2, 3 by Donald E. Knuth 1st Ed
    by Martin Ruckert

13. Art of Computer Programming, Volume 1, Fascicle 1, The: MMIX -- A RISC Computer for the New Millennium 1st Ed
    by Donald Knuth, John Fuller

14. The Art of Computer Programming, Volume 4, Fascicle 5: Mathematical Preliminaries Redux; Introduction to Backtracking; Dancing Links 1st Ed
    by Donald Knuth

15. Art of Computer Programming, Volume 4, Fascicle 6, The: Satisfiability 1st Ed
    by Donald Knuth

Nice to have book that introduce and give already a middle ground on the subject of algorithms, with nice illustrations.

16. Introduction to the Design and Analysis of Algorithms 3rd Ed
    by Anany Levitin

Also a good introduction book to algorithms, with simple explanations and Java code.

17. Algorithms 4th Edition
    by Robert Sedgewick, Kevin Wayne

BioInformatics algorithms

1. BioInformatics Algorithms - An Active Learning Approach, 3rd Ed
   by Phillip Compeau, Pavel Pevzner
   https://www.bioinformaticsalgorithms.org/

Numerical algorithms

In terms of numerical algorithms there are plenty to learn. You can go with a somewhat old book.

1. Numerical Recipes 3rd Edition: The Art of Scientific Computing
   by William H. Press

You will need a bases of numerical methods and it’s applications.

2. Numerical Methods for Engineers 8th Ed
   by Steven Chapra, Raymond Canale

3. Numerical Methods in Physics with Python
   by Alex Gezerlis

4. Computational Physics: Problem Solving with Python 3rd Ed
   by Rubin H. Landau, Manuel J Páez, Cristian C. Bordeianu

5. Applied Computational Physics
   by Joseph F. Boudreau, Eric S. Swanson

6. Hans Petter Langtangen - Various writings
   http://hplgit.github.io/

7. Hans Petter Langtangen - Last versions
   https://library.oapen.org/discover?rpp=10&etal=0&query=Langtangen%2C+Hans+Petter&scope=&group_by=none&page=1

Mathematical bases

For in depth algorithm analysis you will need discrete math, the following is a good book:

1. Discrete Mathematics with Applications 5th Ed
   by Susanna S. Epp

You will need to learn Linear Algebra with examples in code (Python and Matlab) without calculus.

2. Linear Algebra: Theory, Intuition, Code
   by Mike X Cohen

Then for common mathematics you start with a Pre Calculus book like.

3. Precalculus
   by Jay Abramson
   https://openstax.org/details/books/precalculus

Then to my knowledge there are 3 similar good paths that you can follow, but with increasing depth in mathematics and Calculus.

First path

4. Engineering Mathematics, 5th Ed
   by Prof Anthony Croft, Dr Robert Davison, et al.

Second path

5. Modern Engineering Mathematics, 6th Ed
   by Glyn James, Phil Dyke

6. Advanced Modern Engineering Mathematics, 5th Ed
   by Glyn James, David Burley, Dick Clements, et al.

Third path

7. Mathematical Methods for Physics and Engineering: A Comprehensive Guide 3rd Ed
   by K. F. Riley

8. Student Solution Manual 1st Ed for Mathematical Methods for Physics and Engineering 3th Ed
   by K. F. Riley

Then you need to learn about Probability and Statistics the following are two nice books with a companion book with the solutions.

9. Probability: For the Enthusiastic Beginner
   by David J. Morin

10. Introduction to Probability, Statistics, and Random Processes
    by Hossein Pishro-Nik
    http://www.probabilitycourse.com/preface.php

11. Student's Solutions Guide for Introduction to Probability, Statistics, and Random Processes
    by Hossein Pishro-Nik
    http://www.probabilitycourse.com/preface.php

Then you will need to learn about Optimization, two good books, the first with code in Julia.

12. Algorithms for Optimization
    by Mykel J. Kochenderfer, Tim A. Wheeler
    Note: See the book PDF site link on the authors page.
    https://mykel.kochenderfer.com/textbooks/

13. Convex Optimization
    by Boyd, Vandenberghe
    https://web.stanford.edu/~boyd/cvxbook/

In this context...

• See my guide on Mathematics.
  How to learn Mathematics from the ground up
  https://github.com/joaocarvalhoopen/How_to_learn_Mathematics_from_ground_up

All my other guides

• The links to all my guides are in:
  Guides on Linux - Programming - Embedded - Electronics - Aeronautics
  https://github.com/joaocarvalhoopen/Guides_Linux-Programming-Electronics-Aeronautics

Have fun!

Best regards,
João Nuno Carvalho