srpallab / recipe-api-re

Django REST For Recipe App

App Design

19 API Endpoints

• Schema
  • GET /api/schema/
• Health Check
  • GET /api/health-check/
• Managing recipes
  • GET /api/recipe/recipes/
  • POST /api/recipe/recipes/
  • GET /api/recipe/recipes/{id}/
  • PUT /api/recipe/recipes/{id}/
  • PATCH /api/recipe/recipes/{id}/
  • DELETE /api/recipe/recipes/{id}/
  • POST /api/recipe/recipes/{id}/upload-image/
• Managing users
  • POST /api/user/register/
  • POST /api/user/login/
• Managing tags
  • GET /api/recipe/tags/
  • PUT /api/recipe/tags/{id}/
  • PATCH /api/recipe/tags/{id}/
  • DELETE /api/recipe/tags/{id}/
• Managing Ingredients
  • GET /api/recipe/ingredients/
  • PUT /api/recipe/ingredients/{id}/
  • PATCH /api/recipe/ingredients/{id}/
  • DELETE /api/recipe/ingredients/{id}/

Project Structure

• app/ : Django Main Project
• app/core/ : Code Shared Between Multiple Apps (Database Related Codes)
• app/user/ : User Related Codes
• app/recipe/ : Recipe Related Codes-
• Swagger : Browse APIs

Intro to TDD (Unit Test)

• Code which tests code
• Sets up conditions/input
• Runs a piece of code
• Checks output with “assertions”

System Setup

• Docker, Docker compose
• Git

Project Setup

Docker

• Define Dockerfile
• Create Docker Compose configuration
• Run all commands via Docker Compose

Docker Hub introduce rate limit

• 100 pulls / 6hr for unauthenticated users
• 200 pulls / 6hr for authenticated users

GitHub Actions is a shared service

• 100 pulls / 6hr applied for all users

Authenticate with Docker Hub

• Create account
• Setup credentials
• Login before running job
• Get 200 pulls / 6hr for free.

Configure Docker

• Choose Python as base image
• Install Dependencies
• Setup Users

Configure Docker Compose

• Define services
• Name (example:app)
• Port mapping
• Volume mapping
• Command Example

  docker-compose run --rm app sh -c "python manage.py collectstatic"
      

  • docker-compose runs a Docker Compose command
  • run will start a specific container defined in configuration
  • --rm remove the container
  • app is the name of the service
  • sh -c passes in a shell command
  • "python manage.py collectstatic" command to run inside container

Linting

• Install flake8 package
• Run it through Docker Compose

  docker-compose run --rm app sh -c "flake8"
      

Testing

• Django test suite
• Setup test per Django app
• Run tests through Docker Compose

  docker-compose run --rm app sh -c "python manage.py test"
      

Starting Django Project

docker-compose run --rm app sh -c "django-admin startproject app ."

Configure GitHub Actions

• Automation tools
• Similar to Travis-CI, GitLab, CI/CD, Jenkins
• Run jobs when code changes
• Automate tasks

Common uses

• Deployment
• Code linting
• Unit tests

How it works

• Trigger: Push to GitHub
• Job: Run unit test
• Result: Success or fail

Pricing

• Charged per minutes
• 2,000 free minutes

Configure

• Create a configuration file at .github/workflow/checks.yml
• Set trigger
• Add steps for running and linting
• Configure Docker Hub authenticate

Authenticate Docker Hub

• Use docker login
• Add secrets to GitHub project Actions Menu
  • DOCKERHUB_TOKEN
  • DOCKERHUB_USER

TDD with Django

Django test framework

• Based on the unittest library
• Django adds features
  • Test client - dummy web browser
  • Simulation authentication
  • Temporary database
• Django REST Framework adds features
  • API test client

Where do you put tests

• Placeholder tests.py added to each app
• Or, create tests/ sub-directory to split tests up
• keep in mind
  • Only use test.py or tests/ directory (not both)
  • Test modules must start with test_
  • Test directories must contain __init__.py

Test Database

• Test code that uses the DB
• Specific database for tests
• Happens for every test

Test Class

• SimpleTestCase
  • No database integration
  • Useful if no database is required for test
  • Save time executing tests
• TestCase
  • Database integration
  • Useful for testing code that uses the database

Writing tests

• Import test Class
• Import objects to test
• Define test class
• Add test method
• Setup inputs
• Execute code to be tested
• Check output

Mocking

• Override or change behave of dependencies
• Avoid unintended side effects
• Isolate code being tested
• Example:

  register_user() ---> create_in_db() ---> send_welcome_email()
      

• use unittest.mock
  • MagicMock/Mock replace real objects
  • patch override code for tests

Testing API

• Make actual requests
• Check results
• Django REST framework API Client
  • Based on Django Test Client
  • Make requests
  • Check results
  • Override any authentication
• Using the API Client
  • Import APIClient
  • Create client
  • Make request
  • Check result

Setup and Configure Database

PostgreSQL

• Integrate well with Django

Docker Compose

• Define with project (re-usable)
• Persistent data using volume
• Handles network configuration
• Environment variable configuration

Network connectivity

• Set depends_on on app service to start db first
• Docker Compose creates a network
• The app service can use db host name

Volumes

• Persistent data
• Maps directory in container to local machine.

Initial Database Configurations for new DB service

environment:
  - POSTGRES_DB=devdb
  - POSTGRES_USER=devuser
  - POSTGRES_PASSWORD=password

• Creates a DB named devdb
• Creates a user named devuser
• Creates a password for devuser with password

Steps for configuring database

• Tell Django how to connect
• Install the tool Django uses to connect
• Update requirements to install PostgreSQL adapter
• Django settings
  • Engine (type of Database)
  • Hostname (IP or domain name for database)
  • Port (5432)
  • Database Name
  • Username
  • Password
• Pull configuration values from environment variable
• Easily passed to Docker
• Used in local development or production
• Single place to configure project
• Easy to do with Python

psycopg2

• The package that you need in order for Django to connect to our Database
• Most popular PostgreSQL adapter for Python
• psycopg2-binary
  • Okay for development
  • Not good for production
• psycopg2
  • Compiles form source
  • Required additional dependencies
  • Easy to install with Docker

Installing Psycopg2

• List of package dependencies in docs
  • C compiler
  • python3-dev
  • libpq-dev
• Equivalent packages for Alpine
  • postgresql-client
  • build-base (can be cleaned)
  • postgresql-dev (can be cleaned)
  • musl-dev (can be cleaned)

Problem with Docker compose

• Using depends_on ensure service start, doesn’t ensure application is running
• Make Django Management Command “wait for db”
  • Check for database availability
  • Continue when database ready
• Create custom Django management command
• Create new app core for db

  docker-compose run --rm app sh -c "python manage.py startapp core"
      

Create User Model

Django Authentication

• Build in Authentication system
• Framework for basic features
  • Registration
  • Login
  • Authentication
• Integrates with Django Admin

Django user model

• Foundation of the Django Authentication system
• Default user model use username instead of email
• Not easy to customize
• Create a custom model for new projects

Customize user Model

• Base from AbstractBaseUser and PermissionsMixin
• Create custom manager
• Used for CLI integration
• Set AUTH_USER_MODEL in settings.py
• Create and run migrations

AbstractBaseUser

• Provide features for authentication
• Doesn’t include field

PermissionsMixin

• Support for Django permission system
• Include fields and methods for authentication system

Custom issues

• Running migrations before setting custom model.
  • Set custom model first
• Typos in configuration
• Indentation in manager or model

Design custom user model

• User fields
  • email (EmailField)
  • name (CharField)
  • is_active (BooleanField)
  • is_staff (Booleanfield)
• User model manager
• Used to manage objects
• Custom logic for creating objects
  • Hash password
• Methods for Django CLI
  • Create superuser
• BaseUserManager
  • Base class for managing users
  • Useful helper methods
    • noramlize_email for storing emails consistently
  • Custom Methods
    • create_user called when creating user
    • create_superuser used by the CLI to create a superuser (admin)

Setup Django Admin

How to enable Django Admin

• Enable per model
• Inside admin.py
  • admin.site.register(User)

Customizing

• Create class based off ModelAdmin or UserAdmin
• Override or set class variables

     from django.contrib.admin import BaseUserAdmin
  
  
     class UserAdmin(BaseUserAdmin):
  	   """Define the adminpages for user"""
  	   ordering = ['id']
  	   list_display = ['email', 'name']
  	   fieldsets = (
  	       (
  		   None,
  		   {'fields': ('email', 'password')}
  	       )
  	   )
  	   readonly_fields = ['last_login']
  	   add_fieldssets = (
  	       (
  		   None,
  		   {
  		       'classes': ('wide',),
  		       'fields': ('email',),
  		   }
  	       )
  	   )
      

API Documentation

What to document?

• Everything needed to use the API
• Available endpoints (paths)
  • /api/recipes
• Supported methods
  • GET, POST, PUT, PATCH, DELETE
• Format of payloads (inputs)
  • Parameters
  • Post JSON format
• Format of responses (output)
  • Responses JSON format
• Authentication process

Docs in DRF

• Auto generate docs (with third party library drf-spectacular)
• Generate Schema
• Browse able web interface
• Make test requests
• Handle Authentication

How it works

• Generate “schema” file
• Parse schema into GUI

OpenAPI Schema

• Standard for describing API
• Popular in industry
• Supported by most API documentation tools
• Uses popular formats: YAML/JSON

Using a Schema

• Download and run in local Swagger instance
• Serve Swagger with API

View

• Handles a request made to a URL
• Django uses functions
• DRF uses classes
  • Reusable logic
  • Override behavior
• DRF also support decorators
• APIView and Viewsets = DRF base classes
• API View
  • Focused around HTTP methods
  • Class methods for HTTP methods
  • Provide flexibility over URLs and logic
  • Useful for non CRUD API
    • Avoid for simple Create, Read, Update, Delete API
    • Bespoke logic (example: authentication, jobs, external API)
• Viewsets
  • Focused around actions
    • Retrieve, list, update, partial update, destroy
  • Map to Django Models
  • Use Routes to generate URLs
  • Great for CURD operations

Nested Serializers

• Serializer within a serializer
• Used for fields which are object
• JSON Example:

     {
  	  "title": "Some Title",
  	  "user": "Jeff",
  	  "tags": [
  	      {"name": "Tag 1"},
  	      {"name": "Tag 2"}
  	  ]
     }
      

• Python Example:

     class TagSerializer(serializers.Serializer):
  	  name = serializers.CharField(max_length=100)
  
  
     class RecipeSerializer(serializers.Serializer):
  	  title = serializers.CharField(max_length=100)
  	  user = serializers.CharField(max_length=100)
  	  tags  = TagSerializer(many=True)
      

• Limitation
  • Read only by default
  • Custom logic to make write able or edit able

Build User API

Create App in Docker

docker-compose run --rm app sh -c "python manage.py startapp user"

User API

• User register
• Creating authentication token
• Viewing or updating profile

Endpoints

• user/create/
  • POST - Register a new user.
• user/token/
  • POST - Create new token
• user/me
  • PUT/PATCH - Update profile
  • GET - View profile

Authentication

• Basic : Send username and password with each request
• Token : Use a token in the HTTP header
• JSON Web Token (JWT) : Use an access and refresh token
• Session : Use cookies.

Build Recipe API

Features

• Create
• List
• View Detail
• Update
• Delete

End Points

• /recipes/
  • GET - List all recipes
  • POST - Create recipes
• /recipes/<recipe_id>/
  • GET - View details of recipe
  • PUT/PATCH - Update recipe
  • DELETE - Delete recipe

Build Tags API

Tasks

• Add ability to add recipe tags
• Create model for tags
• Add tag API endpoints
• Update Recipe endpoint
  • Adding and Listing tags

Model

• name : Name of tag to create
• user : User who created/owns tag

Tag Endpoint

• /api/recipe/tags
  • GET : List available tags
  • POST : Create tag
  • PUT/PATCH : Update tags
  • DELETE : Remove tag

Build Ingredients API

Ingredient Model

• name: Name of ingredient to create
• user: User who owns ingredient

Ingredient Endpoint

• /api/recipe/ingredients/
  • GET : List Ingredients
• /api/recipe/ingredients/<id>
• GET : Get Ingredient
• PUT/PATCH : Update Ingredient
• DELETE : Remove Ingredient
• /api/recipe/
  • POST : Create ingredient (as part of recipe)
• /api/recipe/<id>
  • PUT/PATCH : Create or modify ingredient

Recipe Image API

• Handling static/media files
• Adding image dependencies
• Update recipe model with image field
• Add image upload endpoint
• Endpoints:
  • /api/recipes/<id>/upload-image
    • POST: Upload image
• Additional dependencies
  • Pillow (Python Imaging Library)
    • zlib, zlib-dev
    • jpeg-dev
• Media and Static: Files not generated by Python Code
  • Images, CSS, JavaScript, Icons
• Two Types:
  • Media : Uploaded at run time (user uploads)
  • Static : Generated on build
• Configurations
  • STATIC_URL: Base static URL (Example: static/static)
  • MEDIA_URL: Base media URL (Example: static/media)
  • MEDIA_ROOT: Path to media on file system (Example: /vol/web/media)
  • STATIC_ROOT: Path to static files on file system (Example: /vol/web/static)
• Docker Volumes:
  • Store persistent data
  • /vol/web : store static and media sub-directories
• Diagram (Deployed)
  • HTTP (GET static....) ----> NGINX ----> Docker (vol/web....)
  • GET : /static/media/file.jpeg <— /vol/web/media/file.jpeg
  • GET : /static/static/admin/style.css <— /vol/web/static/admin/style.css
• Collect Static
  • Django provides command to gather static files
  • python manage.py collectstatic
  • Puts all static files into STATIC_ROOT

Implementing Filtering

• Filter recipe by ingredients / tags
  • Find certain types of recipe
• Filter tags / ingredients by assigned
  • List filter options for recipes
• Define OpenAPI parameters
  • Update documentation
• Example requests
  • Filter recipes by tag(s):
    • GET : /api/recipe/recipes/?tags=1,2,3
  • Filter recipes by ingredient(s):
    • GET : /api/recipe/recipes/?ingredients=1,2,3
  • Filter tags by assigned:
    • GET : /api/recipe/tags/?assigned_only=1
  • Filter ingredients by assigned:
    • GET : /api/recipe/ingredients/?assigned_only=1

Deploy on AWS