GitHub cheat sheets:
https://gist.github.com/TT--/14260aef6c0e31fca5b37e7cb3c53020
Figures with caption in markdown:
<figure><img src="images/myimage.jpg"><figcaption>image caption</figcaption></figure>
https://github.com/maptime/getting-started-with-git-and-github/blob/master/README.md
git add .
git commit -am "I have made some changes."
// git checkout develop
//git merge --no-ff my-feature-branch
git push origin develop
git status
-
1a. Pandas versus NumPy
1b. Spyder versus VS Code
1c. Connecting to a database
1d. Testing frameworks
1di. UnitTest
1dii. PyTest
1diii. Selenium
-
[ ] Install software tools
[ ] put 3-ring binder of relavent papers together
[ ] Look at Testing plans - hardware focused
[ ] Tools/Methods for automated testing/scripting
[ ] notes from testing class- located vol 1 - located: C:\Users\Bob Ross\Desktop\St Thomas Classes\SEIS_627_03_Asgn_2_0005-Heather\DOCs. Find: vol 2
-
3a. Testing Books
3b. Python Books
-
4a. Python & Testing
4b. Unit Testing
4c. Design Patterns for Accessing Hardware
-
5a. Setup methods & Fixtures?
5b. Non-constant static fields?
5c. Data Frames?
5d. What version control?
PEP 8 is a style Guide for writing Python code.
PyLint enforces PEP8 standards. Tools for UML diagrams, refactoring assistance, & IDE integration.
Typora is a simple and configurable document editor that provides excellent Markdown syntaxt support for documentation.
Anaconda Navigator is an data science environment that contains Jupyter Notebooks (beta), Jupyter Lab (final release) for documentation, conda environment, R Studio, Sypder IDE (data display is better and cleaner, JSON, dataFrames) and Visual Studio Code IDE.
Support for interactive data visualization inside Jupyter Notebook.
JupyterLab is Jupyter’s Next-Generation Notebook Interface. JupyterLab is a web-based interactive development environment for Jupyter notebooks, code, and data. JupyterLab is flexible: configure and arrange the user interface to support a wide range of workflows in data science, scientific computing, and machine learning. JupyterLab is extensible and modular: write plugins that add new components and integrate with existing ones.
Pandas is a fast, powerful, flexible open source data analysis and manipulation tool, built on top of the Python programming language. Pandas provides a bunch of additional data structures (See here). Intro to pandas here. Installation instructions here.
NumPy is a library for Python that adds support for large, multi-dimensional arrays and matrices, along with a large collection of high-level mathematical functions to operate on these arrays. Pandas is a high-level data manipulation tool that is built on the NumPy package. Think of this like the PVC pipe that connects all the different libraries in python? NumPy arrays are very fast compared with native datatypes in python.
- Opinion #0 - Opion #1 - Opinion #2 -
Spyder is more geared towards tracing data and might be preferred. Its well suited to JSONs and data frames.
PyCharm and Visual Studio Code (VS Code) are both very popular and featureful IDEs. However, PyCharm is tailored for Python and you'll have to install other IDEs for other languages. Visual Studio is more modular, it only needs to know what type of project you are working in and it will automatically enable the required plugins. It’s worth noting that PyCharm is very resource-intensive and performs much slower than Visual Studio. Consider PyCharm if you'll be working mainly with Python. Consider Visual Studio if you need to work with multiple languages and want the flexibility to customize your IDE.
Testing Frameworks include: 1.) unitTest; 2.) Nose (no longer supported); 3.) pytest; and, 4.) Avacado. Selenium is a common tool for test automation.
PyTest and UnitTest are the most popular frameworks. UnitTest is built into the standard python library. PyTest might come pre-installed. Basically everything in PyTest is done with Assert - very clean API. UnitTest has a more complicated API. docTest is an object that might be useful within the python libraries.
File paths/file names - min 7, G video 1
-
Python Tutorial: Unit Testing Your Code with the unittest Module video (39 min)
-
UnitTest, like pyTest, does test discovery.
-
Logging unit tests: How to: Output debug information during unit tests, Logging Docs, unitTest + html report video (1 hour)
-
Installation instructions here. API reference here. Tips here. Pytest has never been associated with a security vulnerability, however, PyTest Security vulnerabilities are reportable here.
-
Python Testing 101 with PyTest video
-
Automated testing with pytest and fixtures video (27 min)
xdist is a tool that lets you run all tests in parallel.
Syntax: Test functions are named like
test_myTestFunctionand classes containing tests are named likeTest_myTestClass"Test Discovery" - Regular expressions are used to run multiple tests sets in command line. (min 3) This is highly configurable and you can make it work many different ways. Regexes help you run subsets of tests by names.
pytest file test_primes.py will produce the following output if all 5 tests pass:
test_primes.py .....and will produce the following output if two tests failed:test_primes.py .F.F. -
Python Testing 101 with pytest video (70 min). Python Testing 201 with pytest (60 min)
-
Course on pytest and selenium: Tutorial 1, Tutorial 3
[x] Quick write up of potential tools
[ ] Install software tools
[ ] put book of papers together
[ ] Look at Testing plans - hardware focused
[ ] Methods for automated testing/scripting
[ ] notes from testing class- located vol 1 - located: C:\Users\Bob Ross\Desktop\St Thomas Classes\SEIS_627_03_Asgn_2_0005-Heather\DOCs. find: vol 2
[x] call recruiter Monday to verify adequate paperwork was received. Background check successful. Labwork is stuck. Labs are overloaded.
[x] push notes to github
[x] Assemble a little library - call half price books - picked up python book
-
Testing Computer Software (2nd edition) by Cem Kaner, Jack Falk, Hung Q. Nguyen. ISBN-13:978-0471358466. Link to amazon
-
Software Quality Assurance and Evaluation First Printing Edition by James H. Dobbins. 1990. ISBN-13: 978-0873890595. Link to amazon
-
Automate the boring stuff by Al Sweigart. Free online.
-
Python Crash Course (more verbose version of previous book)
-
-
Effective Python - I bought the 1st edition at Half Price, but there is a 2nd edition.
-
Learning python by Mark Lutz
- Using Python to Test - Sep 9, 2015 – 26 min video
Tools & methods to establish a python test framework for hardware models. Broad overview.
This man recommends using decorator pattern as a base for swapping hardware in and out of a system. I'm not sure that I'm in love with this. I'm looking at Abstract Factory Pattern - a form of Factory Pattern. There are 3 versions of this pattern: 1.) Simple factory; 2.) Abstract Factory; and 3.) Factory method.
- Andrew Knight | Testing is Fun in Python! - Oct 4, 2016 - 40 min video
Good testing principals & 5 Testing frameworks: 1.) unitTest; 2.) doctests ; 3.) Nose ; 4.) pytest; 5.) Avacado.
-
Unit Testing Video
-
Effective Unit Testing - Apr 11, 2017 - 48 min video
Fundamental Principal of Unit Testing (min 5), characterization tests. Test should only be coupled to the API, not the implementation. Tests written first should all fail before you write your implementation. 1 test has 1 and only 1 assertion. Unit tests are totally independent (Tests can run in parallel in multiple threads - don't have any order to how the tests can run). Don't depend on semaphores or synchronization or special data structures. Don't use non-constant static fields in tests. If model code under test has global state - beware. Tests DO NOT share instance data. Setup determines if you need a new test class. Run slower tests last.
A single test should 1 in 1 sec or less. A test suite should run in 1 min or less.
Passing tests should produce no output. (min 24)
Rotate your test data - don't use the dame data every time.
Flakiness stems from: Time dependence, network availability, Explicit randomness, multithreading (race conditions - record your data dumps. They fail periodically). Tests can deadlock. (min 30)
System skew - The tests pass every time I run them, but fail when someone else runs them. This is more common when using C and C++. Assumptions were made re: the underlying operating system. Floating point roundoff errors. Integer width. DONT USE THE DEFAULT character set ever. Model code should define it as UTF-8.
Anti-Pattern: Conditional logic in tests. If else statements are bad because they should be divided into 2 separate test methods. Its easy for these test to not run and you to not get the test coverage you intend. Fixed input, fixed output ONLY.
Don't fix bugs. Write it up and file the error report. Iterate on the test. That's how you find a bug. You should see the failed test identifying the bug pop up in the documentation. Write bug tests first.
Refactoring (min 37)
Sometimes your tests aren't actually running on the code, so you start by writing a failing test. Write additional tests before you do any refactoring.
Continuous Integration (min 39) - submit queue - Never directly merge your own code. It commits iff all the tests pass.
Never, ever merge/commit code that has a failing test.
All test are reproducible and unambiguous.
Chapter 3: Design Patterns for Accessing Hardware – Design Patterns for Embedded Systems in C - June 21, 2021 - 144 pages. 3.1: Basic Hardware concepts
Hardware/Device Design Pattern
https://hackaday.com/2018/01/12/software-design-patterns-for-real-hardware/
What quantitative measures can be collected?
[ ] Tools and measures: https://www.guru99.com/code-coverage.html
[ ] MC/DC: https://www.youtube.com/watch?v=DivaWCNohdw
[ ] https://www.rapitasystems.com/mcdc-coverage
[ ] https://ntrs.nasa.gov/api/citations/20040086014/downloads/20040086014.pdf
[ ] https://www.researchgate.net/publication/269168778_An_Automated_Tool_for_MCDC_Test_Data_Generation
[ ] NISTS open source MCDC tool & paper: https://www.nist.gov/publications/mcdc-star-150-open-source-mcdc-measurement-tool
[ ] paper/tool: https://ieeexplore.ieee.org/document/8563199
Setup methods & Fixtures in pyTest
[Automated testing with pytest and fixtures Video (27 min) ][(https://www.youtube.com/watch?v=8mp_1Jt-xHQ)]
Non-constant static fields ( See min 19 of Effective Unit Testing video)
Data frames are matrices or tables. DFs are a data type available from inside the pandas package. These were created by Wes McKinsey and use their own goofy syntax. Knowing python will not help you with DFs. Useful functions include "slicing" (see G video 1 - min 12) - print stringy([:2]) will print all elements from 0 to 1. When using data frames, we would instead write print(df.iloc[:2])to print the index locations inside the data frame. DataFrames: The Good, Bad, and Ugly (20 min)
What version control are we using?